Oral hygiene system for compliance monitoring and tele-dentistry system

ABSTRACT

Oral hygiene monitoring system is disclosed to track motion and orientation of an oral hygiene device. The control system may process data output from a motion sensor to determine position and orientation of an oral hygiene device with respect to a user&#39;s mouth. In some examples, the system may use machine learning techniques to identify position and orientation, brush stroke type, and other specific actions.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The application claims the benefit of and priority to U.S. provisionalpatent application no. 62/378,139, filed Aug. 22, 2017, which is herebyincorporated by reference herein in its entirety.

FIELD

The present invention relates to methods and devices for monitoring oralhygiene activities.

BACKGROUND

Compliance with proper technique and frequency of oral hygieneactivities, including brushing and flossing, is essential for healthyteeth. However, compliance is especially poor among children andadolescents. For instance, many regions of the mouth are frequentlymissed after bad habits develop. Accordingly, if the amount of brushingand technique could be monitored, compliance perhaps could be increased.However, monitoring the oral hygiene device technique and the amount oftime in each region is quite difficult due to various technologicallimitations.

According to the CDC, although preventable, tooth decay is the mostcommon chronic disease of children aged 6-11 (25%) and adolescents aged12 to 19 years (59%). Also 28% of adults aged 35 to 44 have untreatedtooth decay. A bacterial biofilm called plaque that develops on teethcontributes to tooth decay and gingivitis. However, plaque can beremoved by brushing at least once a day for two minutes and preferablytwice a day and therefore prevent or mitigate tooth decay. Atlin T andHorecker E., “Tooth Brushing and Oral Health: How Frequently and WhenShould Tooth Brushing be Performed,” Oral Health & Prevention Dentistry,2005 3 (3): 135-140.

Additionally, research shows that children continually miss the sameareas during brushing which leads to isolated buildups of plaque oncertain teeth. Accordingly, more important than the length of time ofbrushing, is the efficacy of the tooth brushing. Additionally, dentalhealth education only has been shown to generally only have a small andtemporal effect on plaque accumulation. Atlin T and Horecker E., “ToothBrushing and Oral Health: How Frequently and When Should Tooth Brushingbe Performed,” Oral Health & Prevention Dentistry, 2005 3 (3): 135-140.Furthermore, many toothpastes incorporate fluoride which promotes theregrowth of tooth enamel to prevent cavity formation. According to theAmerican Dental Association, the compliance with tooth brushing is quitelow. For instance, only 49% of men and 57% of women brush their teethtwice a day.

SUMMARY

Accordingly, there exists a need for a dental system that could increasea user's compliance of tooth brushing, flossing, or other oral hygieneactivities with dentist recommend regimes in order to decrease cavities,gum disease, and other dental complications from lack of brushing. Thepresent disclosure provides systems and methods for monitoring oralhygiene device usage and electronically providing feedback and otherincentives to the user to increase compliance.

The system allows the recording of teeth brushing activities (or otheroral hygiene activities) and the analysis of those activities. It allowsthe creation of a service that provides feedback and incentives for auser of the oral hygiene system. The oral hygiene device or associatedsystem components, including for instance, a camera, records brushingdata through sensors during its use. In some embodiments, the data isanalyzed and compared to reference data. For instance, one or moreoptical sensor(s) are disclosed that track the movement of thetoothbrush with reference to the mouth. Images output from the opticalsensor(s) may be analyzed to determine an orientation and motion of thetoothbrush with respect to the mouth, and amount of brushing in eachregion (or flossing with, for example, a water flosser) may bedetermined.

Through a feedback output device, the user receives advice about the useof the system and incentives and other feedback designed to increasecompliance with recommended usage regimes. For example, the system canlet the user know which regions of the mouth were brushed or flossed andprovide the user feedback for where they could brush more, or positionsor portions of the mouth they entirely missed. In another example, thesystem may implement a gamification process to increase the motivationto use the hygienic device.

In some embodiments, the invention relates to a method for a new way touse an oral hygiene device by informing a user about his/her brushingpractices for example by wireless integration with a mobiletelecommunication device or other device having a display. The electrictooth brush may also communicate data wirelessly to a base station whichmay then send the data to a network for analysis on cloud servers orwirelessly to a mobile device. The mobile telecommunication device maybe a mobile phone, a microcomputer with telecommunication means, atablet computer with telecommunication means. In other embodiments, thedata may be wirelessly sent to the base station and then uploaded toservers for later accessing by computing devices that include bothmobile and non-mobile computing devices. In some examples, the oralhygiene system will include one or more optical sensors or cameras thattrack the movement of a manual toothbrush (e.g., anon-powered/non-electric toothbrush), and the camera will integrate witha mobile device, a base station, a local area network, or othercomputing devices. In this example, any standard toothbrush or waterflosser may be utilized that does not include electronics or motionsensors.

The oral hygiene device system may include an oral hygiene device withsensors and a base station, the base station physically supporting theoral hygiene device when it is not handled by a user. The electronicoral hygiene device may include (a) signal processing circuitry, (b)memory, (c) base station interface for exchanging data between the oralhygiene device and base station, (d) a power supply circuit that mayinclude a rechargeable battery or capacitor, and (e) a controller.

A base station and/or camera may include (a) a network interface forexchanging data between the internet or other network and the basestation, and (b) a recharging circuit for recharging the toothbrush'srechargeable battery which may optionally also act as a magnetictransmitter in connection with a magnetometer sensor in the toothbrush.In other embodiments, the system may not include a base station andsignals may be sent wirelessly directly to a mobile phone or otherwireless terminal, or a separate optical sensor/camera system may recordthe optical data and send it directly to a mobile device, othercomputing device or network for analysis. In some embodiments, the basestation may include a camera for monitoring and identifying codes on theoral hygiene device to visually track movement.

The oral hygiene device system may include a camera located separatefrom the oral hygiene device to record images of the oral hygiene deviceduring brushing. Image processing software can then analyze the brushingmotion of the oral hygiene device independently or in addition to datafrom motion sensors attached to the toothbrush. For instance, the oralhygiene device may not incorporate any electronics and may instead be astandard toothbrush. In this example, the camera and image processingsystem may solely determine the brushing position and times for eachsection of teeth.

For instance, the optical sensor(s) may record images during the entirebrushing session that include the mouth, oral hygiene device and teeth.The image processing software may then identify the toothbrush, featuresof the toothbrush, the mouth, and the relative positions of each todetermine the section and time of brushing, or using other methods asdiscussed further herein.

In some examples, the oral hygiene device may include a pattern forenhanced recognition and spatial orientation calculation. In someexamples, an attachment for an oral hygiene device may also contain orinclude a pattern. Additionally, the system may combine the sensormotion data output from electronics on the oral hygiene device withimage data to determine the section or portion of teeth a user isbrushing.

Electrical Configuration

The control system(s) of the oral hygiene devices may be configured tocoordinate the data exchange between the oral hygiene device, opticalsensor(s), base station, mobile device and/or other networked devicesfor the transfer of the processed signals from the sensors and/oroptical sensor for processing to the processing. In some embodiments,the oral hygiene device system and associated control system may includesignal conditioning circuits for the processing of signals from thesensors, a memory for the storing of the processed signals from thesensors, an oral hygiene device interfacing circuit for allowinginformation exchanges between the oral hygiene device and the basestation and other electronic components, an oral hygiene device powersupply circuit for powering the sensors and the circuits of thetoothbrush, including a rechargeable electric source of the batteryand/or capacitor type, and a controller circuit for directing theoperation of the tooth brush electronics.

The base station, optical sensor(s), and/or other associated cameradevice may include the following circuits: a data exchange circuit forexchanging data with a network, an interfacing circuit adapted toexchange information with the an oral hygiene device, a base stationpower supply circuit for powering the base station circuits and forrecharging the rechargeable electric source of the oral hygiene devicewhen it is received in the base station. The base station may alsoinclude a magnetic field transmitter, which may be the power supply orrecharging circuit, or may be a separate magnetic field transmitter. Thebase station or other electronic device may also include a camera andassociated electronics. The controlling circuit may be configured tostore in memory the processed signals from the sensors upon thedetection of the user using the toothbrush, and to command, when theoral hygiene device is received in the base station, the data exchangecircuit of the base station or other electronic device to transfer thestored signals from the sensors, through the interfacing circuits of theoral hygiene device and of the base station or other electronic device,over the network. In some embodiments, raw data from the sensors may bestored and sent over the data exchange circuit for processing at thebase station or processing elsewhere.

Physical Design

In some embodiments, the oral hygiene device may include an electronicmotor, for vibrating the oral hygiene device head during brushing.Additionally, the head of the oral hygiene device that includes thebristles or water flosser, may be removably connectable to a body orhandle of the oral hygiene device, and be configured for theinterchangeability of multiple heads. The oral hygiene device may bewaterproof. In some embodiments, a base station may be configured tophysically receive only one, two, three, four or five, or additionalnumbers of toothbrushes or other oral hygiene heads.

In some embodiments, the oral hygiene device may include visual codes orpatterns that may be detected and tracked by a camera or visual baseddetector. For instance, the oral hygiene device head may include apattern and the handle may include a pattern. In some embodiments, thehandle may include an add-on bulb or protrusion on the end that includesa larger pattern for detection by the camera. This “bulb or protrusion”might be a dedicated add-on clipped on the handle that is notnecessarily part of the toothbrush.

Also, this add-on may also be a stand for the toothbrush. For instance,the bulb could ballast the oral hygiene device with water. Accordingly,the add-on could be an empty half sphere with water or other heavysubstance in the bottom. This could allow for the novelty for a user toset down the oral hygiene device at many angles and/or orientations andthe oral hygiene device would stand erect regardless of the angle and/ororientation at which the oral hygiene device is set down.

Sensors

Sensor(s) may include one more optical sensors that are separate fromthe oral hygiene device that can record images of the oral hygienedevice as it is being used. If two or more optical sensors are used, theimages can be utilized to stereoscopically track movement and distanceof the oral hygiene device.

In embodiments where the oral hygiene device includes electronics, thesensors of the oral hygiene device may be one or more of: i) a pressuresensor (10) motion sensors (11), or ii) any other type(s) of sensorscapable of measuring brushing activities of the toothbrush, such as anaccelerometer or an inertial sensor. This may include accelerometers,magnetometers, and gyroscopes, and/or gyrometers. In some embodiments,the oral hygiene device may include at least a pressure sensor and atleast one acceleration sensor. In some embodiments, the processedsignals from the sensors are transferred to the server via a networkthrough the base station. Additionally, processed signals from thesensors are transferred to the server via a network through the mobiledevice.

In other embodiments, the oral hygiene device may have patterns forrecognition by visual based sensors that are stationary and remote fromthe toothbrush, or may be only a standard toothbrush. For instance, amobile phone camera or a camera in the base station may be utilized asan optical sensor to monitor the orientation and position of thetoothbrush. In these embodiments, the oral hygiene device may alsoinclude motion sensors or in other embodiments may have no electronicsto save on the cost of manufacturing. Instead, the oral hygiene devicemay only have patterns for recognition by the camera, or may have nopatterns and the image processing system may recognize the shape, axisand orientation of the oral hygiene device as disclosed further herein.

Computing Devices

In some embodiments, the mobile telecommunication device is a mobilephone, a microcomputer with telecommunication means, a tablet computerwith telecommunication means, or any other means having display meansfor displaying information related to a tooth brushing activity andhaving circuits for a connection to the global network and forcommunicating with the global network. For example, the mobile devicecan typically be a mobile phone, but may also consist of other portablemobile PDA device types (“PDA”) or otherwise, with capacity of radiocommunication or, even, a microcomputer laptop or desktop withtelecommunication means, a tablet computer with telecommunication means.In other embodiments, the signals are viewable on a stationarycomputable device that accesses the data via cloud servers.

Signal Processing

In some embodiments with electronics in the toothbrush, a control systemof the oral hygiene device is configured to store processed signals fromthe sensors in memory. In some embodiments, the control system mayinstead store raw data from the sensors in the memory for sending to abase station or other component of the system where the raw sensor datamay be processed. The control system coordinates the acquisition,processing, and storage of signals once the control system and/or otherprocessor in the system determines a user initiates brushing. In otherembodiments, the control system may coordinate the storing and sendingof raw data for processing elsewhere. Next, the control system may beconfigured to send the stored signals wirelessly to the base station orthe computing device for further processing or initial processing,display, or analysis. In some embodiments, the oral hygiene device maynot contain a controller or any electronics.

Signal processing may include filtering, amplification, conversion,signal conversion from analog to digital, digital filtering, digitaldata compression, digital data reduction, digital data computation, anddigital data conversion. This may be performed at several differentaspects of the system including the oral hygiene device, the basestation, an associated mobile phone, a server linked by a network to thesystem or other locations.

Data Protocols and Transfer

In some embodiments, the oral hygiene device interface circuit and thebase station and/or camera device interface circuits are wirelesscircuits, for example: WiFi®, Bluetooth®, GSM/UMTS and derivatives. Insome embodiments, the data exchange circuit of the base station uses awireless protocol, for example: WiFi®, Bluetooth®, GSM or others. Insome embodiments, the oral hygiene device may have a unique identifier,to allow the pairing of a mobile device and the toothbrush.

In other embodiments, the oral hygiene device interface circuit and thecamera/base station interface circuit may utilize wired connections. Forexample, the data exchange circuit connection to the network is wired.Identification data may be incorporated in the data packets that includethe stored signals from the sensors that are sent over the network. Theidentification may include a serial identity number of the oral hygienedevice or head, a serial identity number of the base station, or anetwork address of the base station. Additionally, tooth brushingmonitoring data obtained during the measuring step may be time-stampedusing data from a oral hygiene device internal clock.

In other embodiments, the network comprises at least a wireless localarea network (WLAN) and during the step of communication, the oralhygiene device transmits data to said mobile device via said WLAN. TheWLAN may operate according to a communication protocol selected from theWi-Fi or Bluetooth protocols. A mobile, camera, or other computingdevice may also be in communication with the local wireless local areanetwork and in the communication step, the tooth brush transmits saiddata to the mobile device via said wireless LAN.

The LAN may include a server that communicates with at least thetoothbrush, and in the communication step, the oral hygiene device maytransmit said data to the mobile device by means of the server. Thetelecommunication network may further comprise a network of separateremote wireless LANs, the server communicating with at least one servervia said remote network, the mobile device also communicating with saidserver via the remote network.

The information exchanged between the oral hygiene device, camera,optical sensor(s) device, mobile device, and/or the base station throughthe interfacing circuits may include data or commands, the dataincluding stored, processed signals from the sensors or raw data fromthe sensors. Information may be transmitted from the oral hygiene deviceto the base station and, conversely, from the base station to thetoothbrush, as needed. The data can also be a program or software updateto store and/or execute by the toothbrush. For example, updates and newfirmware may be wirelessly downloaded and installed on the toothbrush.

Cloud Server or Local Network Processing of Data

In some embodiments, the system includes a server and the stored,processed signals from the sensors or raw data from the sensors(including optical sensors) are transferred over the network to saidserver, the server including storing means for the transferred processedsignals and including computational components under the control of aprogram or software instructions. The program has instructions that areconfigured to send, at the end of the transfer, an erase command overthe network to the oral hygiene device to erase the signals stored onthe oral hygiene device that have been transferred to the server. Theserver may also determine the location of the oral hygiene device usinggeo-location capabilities of the remote network and/or of the mobiledevice.

The server includes memory for storing a history of the successivetransferred stored and processed signals from the oral hygiene devicesensors and/or raw data from the sensors or associated camera(s) andother devices. The program for controlling the computational componentsof the server includes software instructions for analyzing and comparingthe stored and processed signals and to provide computational resultsfrom said analysis and comparison. The program for controlling thecomputational components of the server may include instructions formaking the results of the analysis available to a variety of computingdevices, including a mobile or stationary device, by accessing theserver through an internet page or other variety of methods. Thecomputational analysis from the server may be transferred or downloadeddirectly to a computing device via a network link, which may be madethrough a dedicated communication equipment POA link to the basestation. Accordingly, the computational results from the server aretransferred to the mobile device via a network through a dedicatedcommunication equipment POA, and then to the base station and/or theoral hygiene device via links. When the oral hygiene device is operativeand communicating with the server, said server can update the softwareand/or the parameters running and/or used in the oral hygiene device.Similarly, the server can update the application or parameters relatedto the oral hygiene device and which is running on the mobile device.

User Profile

The system may include a stored user profile associated to the toothbrushing (or other oral hygiene) activity and its related data comprisesthe age, size and gender of the user. During or after the step ofmonitoring oral hygiene (e.g. tooth brushing) activities, the oralhygiene device, mobile device, or server automatically seeks to matchthe user with at least one user profile using at least one predeterminedrule depending on the user profile and of past data. If the user is nota regular user of the oral hygiene device, said user identifies him/heras a guest on the mobile device.

In a step of user identification, a specific user may be associated withthe oral hygiene device and presumed to be the user. If multiple usersfor a given oral hygiene device are utilized, to associate a user with abrushing activity at least in the oral hygiene device and possibly inthe mobile device and/or the server at least for reference purposes forthose last two.

In embodiments that utilize a camera, the user profile may have apicture of the user uploaded or associated with the profile. This willallow the visual based recognition system to automatically determine aspecific user associated with the profile.

Data Output

In some embodiments, the oral hygiene device transmits data in real timeto the mobile, camera, optical sensor device, or other computing device,and in the display step, the computing device displays in real time onthe screen, information related to said data, for example theinstantaneous progress of a user for a brushing episode in progress.

Calibration/Initialization

In some embodiments, the oral hygiene device and/or camera may becalibrated, either in the factory or by the user or both. If acalibration step is performed by the user, the user can be guided inthis process by information given by e.g. the display of the mobiledevice.

Particularly, the visual tracking system may be calibrated by the userwith prompts from the base station or an associated electronic device,(e.g. mobile and/or optical sensor(s)). The prompts may be audio or comethrough a user interface. The calibration program may request the userstand at a certain distance with the oral hygiene device and move aroundin brushing positions until certain lights or audible tones areindicated. The system may calibrate an oral hygiene device with orwithout patterns, and with or without motion sensing electronics.

This calibration system may detect the oral hygiene device size andshape and the size and shape of the user's head, including the distancethat the user is standing away from the camera. In some embodiments, theuser may hold up a standard oral hygiene device sized calibration toolto allow the camera (or other optical sensor) to appropriately calibratethe distance. In some embodiments, the motion data may be combined withthe image data to calibration the system simultaneously.

In a step of initialization, the system may allow for the selection ofthe desired local wireless network or mobile device. This can be doneautomatically or with the help of the user, and these operationscorrespond to a network pairing between the elements of the system whichcommunicate between them.

In some embodiments, the oral hygiene device may include at least apressure sensor, at least one acceleration sensor, signal conditioningcircuits for the processing of signals from the sensors, a memorycircuit for the storing of the processed signals from the sensors, anoral hygiene device interfacing circuit for allowing informationtransfer between the oral hygiene device and a base station, an oralhygiene device power supply circuit for powering the sensors and thecircuits of the toothbrush, said oral hygiene device power supplyincluding a rechargeable electric source of the battery and/or capacitortype, and a controlling circuit for the operation of the oral hygienedevice circuits. Preferably, the controlling circuit is configured tostore in the memory the processed signals from the sensors upon thedetection of the user using the toothbrush, and to command, when theoral hygiene device is received in a base station, the data exchangecircuit of the base station for the transfer of the stored, processedsignals from the sensors, through the interfacing circuits of the oralhygiene device and of the base station, over a network.

According to the invention, the base station (and/or optical sensordevice) includes a data exchange circuit with a network, a base stationinterfacing circuit adapted to exchange information with an oral hygienedevice and/or mobile device or other computing device, a power supplycircuit for powering the base station circuits and for recharging therechargeable electric source of the oral hygiene device when it isreceived in the base station.

From another point of view of this same invention, the inventionprovides a system and also a method using the system for monitoring oralhygiene (e.g. brushing) activities, allowing communications with amobile communication device having a display and information entrymeans, the oral hygiene device and the mobile device communicating bothby radio with a telecommunications network, said telecommunicationsnetwork being adapted to make communicate at least the mobile devicewith at least one further telecommunications device, the methodcomprising the steps of: (a) at least one tooth brushing or oral hygieneactivity monitoring step in which the oral hygiene device performs atleast one measurement indicative of at least the brushing or other oralhygiene activity of the user, (b) at least one stage of two-waycommunication during which: i) a user enters information in the mobiledevice, and the mobile device transmits said information to the oralhygiene device through the telecommunication network, and ii) the oralhygiene device transmits to the mobile device, the data according tosaid tooth brushing activity monitoring measurement, through thetelecommunications network, and (c) at least a display or feedback stepin which the mobile device displays on its screen information based ontooth brushing (or other oral hygiene) activity monitoring datatransmitted in the communication step. The displayed information may berepresentative of an output of a computation done on the transmitteddata, said computation being done in the mobile device or in a server.

The present disclosure provides a product and service which improves theuse and/or motivation for a user to brush their teeth or perform otheroral hygiene activities. Feedback is provided through an output devicelinked with the system. The system provides an oral hygiene deviceincluding sensors which monitor a user's brushing activities andprovides feedback related to the brushing to the user. In someembodiments, a server may provide applications that use and process thedata received from the sensors to provide output data relating to thequality of the brushing and other analysis. Finally, the applicationsrunning on the server may process this output data to provide feedbackto the user to provide a playful/gameful dimension to increase themotivation to brush or floss the user's teeth.

According to some embodiments, an oral hygiene system for monitoringcompliance with an oral hygiene regimen includes an oral hygiene device,an optical sensor, a memory, and a control system. The oral hygienedevice includes a handle and a head. The memory contains machinereadable medium comprising machine executable code having stored thereoninstructions for performing a method of recognizing specific actions ofthe oral hygiene device. The control system is coupled to the memory andincludes one or more processors coupled to the memory. The controlsystem is configured to execute the machine executable code to cause theone or more processors to: (i) receive data output by the opticalsensor, (ii) process the data using a machine learning model todetermine a specific action of the oral hygiene device represented bythe data, wherein the specific action includes an orientation and motionof the oral hygiene device, and (iii) output data representing thespecific action.

According to some embodiments, an electronic oral hygiene system formonitoring brushing and compliance with a tooth brush regime includes anoral hygiene device, a motion sensor, a memory, and a control system.The oral hygiene device includes a handle and a head. The motion sensoris configured to output motion data related to motion of the oralhygiene device. The memory is for storing the data output by the motionsensor and is in data communication with the motion sensor. The controlsystem determines the position of a head of the oral hygiene device in auser's mouth by comparing the motion data to previously recordedcalibration data using a machine learning model.

According to some embodiments, an oral hygiene system for monitoringbrushing and compliance with a tooth brush regime includes an oralhygiene device, a sensor, a memory, and a control system. The oralhygiene device includes a handle and a head. The memory contains machinereadable medium including machine executable code having stored thereoninstructions for performing a method of determining a position andorientation of the toothbrush. The control system is coupled to thememory and includes one or more processors. The control system isconfigured to execute the machine executable code to cause the controlsystem to determine a specific action performed by the oral hygienedevice based at least on data output by the sensor including motion datarepresenting the motion of the oral hygiene device. The specific actionis determined by processing the motion data with a machine learningmodel.

According to some embodiments, a non-transitory, computer-readablestorage medium has stored thereon instructions for performing a methodcomprising machine executable code which when executed by at least onemachine, causes the machine to: (i) determine a spatial position andorientation of an oral hygiene device based on at least data output byan accelerometer comprising motion data representing the oral hygienedevice using a machine learning algorithm, (ii) determine a section of auser's teeth being brushed by the oral hygiene device based on themotion data, and (iii) output an indication of the user's brushing on adisplay based on the determined spatial position, orientation, andsection of the user's teeth.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described in relation to an exemplifiedembodiment and to the following Figures:

FIG. 1 is a schematic view of an example of the system of the invention;

FIG. 2 is a diagrammatic view of an example of the oral hygiene deviceand of an example base station of the system;

FIG. 3A is a perspective view of an example of an oral hygiene deviceand head in accordance with one embodiment of the present invention;

FIG. 3B is a perspective view of an example of an oral hygiene devicehandle;

FIGS. 3C-3E are perspective views of an example replaceable heads thatmay be attached to a head interface;

FIG. 4 is a flow chart illustrating an example of a process utilized inthe present disclosure to record brushing or other hygiene data andprovide feedback to the user;

FIG. 5 is diagram illustrating an example of an oral hygiene device andbase station, with a magnetic field generator in the base station orother associated device;

FIG. 6A is a perspective view illustrating an example of an oral hygienedevice with visual patterns in accordance with one embodiment of thepresent invention;

FIG. 6B is a perspective view illustrating an example of an oral hygienedevice head with visual patterns in accordance with one embodiment ofthe present invention;

FIG. 7 is a perspective view illustrating an example of a system thatidentifies the position of the oral hygiene device visually without apattern;

FIG. 8A is a perspective view illustrating an example of a toothbrushhandle with an insert;

FIG. 8B is a perspective view illustrating an example of a toothbrushwith an insert removed;

FIG. 8C is a perspective view illustrating an example of an insert thatincludes a battery and electronics for a toothbrush;

FIG. 9A is a front view illustrating an example of a mobile devicedisplay with brushing feedback; and

FIG. 9B is a front view illustrating an example of a mobile devicedisplay with brushing feedback.

DETAILED DESCRIPTION

Various examples of the invention will now be described. The followingdescription provides specific details for a thorough understanding andenabling description of these examples. One skilled in the relevant artwill understand, however, that the invention may be practiced withoutmany of these details. Likewise, one skilled in the relevant art willalso understand that the invention can include many other obviousfeatures not described in detail herein. Additionally, some well-knownstructures or functions may not be shown or described in detail below,so as to avoid unnecessarily obscuring the relevant description.

The terminology used below is to be interpreted in its broadestreasonable manner, even though it is being used in conjunction with adetailed description of certain specific examples of the invention.Indeed, certain terms may even be emphasized below; however, anyterminology intended to be interpreted in any restricted manner will beovertly and specifically defined as such in this Detailed Descriptionsection.

Particular implementations of the subject matter have been described.Other implementations are within the scope of the following claims. Insome cases, the actions recited in the claims can be performed in adifferent order and still achieve desirable results. In addition, theprocesses depicted in the accompanying figures do not necessarilyrequire the particular order shown, or sequential order, to achievedesirable results.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations may be depicted in the drawings in aparticular order, this should not be understood as requiring that suchoperations be performed in the particular order shown or in sequentialorder, or that all illustrated operations be performed, to achievedesirable results. In certain circumstances, multitasking and parallelprocessing may be advantageous. Moreover, the separation of varioussystem components in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

FIG. 1 illustrates an overview of the disclosed oral hygiene devicemonitoring and feedback system 100 that includes: an oral hygiene device1 equipped with sensors, a base station 2 for receiving and charging theoral hygiene device 1, a mobile device 30 that wirelessly receives/sendsdata, a dedicated wireless link POA 8, a server 4 and a network 3 fortransferring the information from the server or between other variouscomponents of the system 100.

Data Communication

The oral hygiene device 1 may have an antenna 5 and transceiver meansfor radio communication to a compatible complementary antenna 5 andtransceiver means of the base station 2 through a radio link 31. Theradio-communication link 31 may be for example WiFi or GSM or Bluetoothor their derivatives or other proprietary protocols. Additionally, oneor more optical sensors 9 may communicate with a mobile phone 30, basestation 2, server 4, or other associated computing device as disclosedherein.

In another embodiment, antennas and transceiver means are replaced orcompleted by wired connections or connectors to allow the exchange ofinformation between the oral hygiene device 1, optical sensor 9, and/orthe base station 2. Wired connectors may also provide electric powersupply from the base station to the oral hygiene device 1 for recharginga rechargeable electric source of the latter. In another embodiment, theelectric power supply from the base station to the oral hygiene device 1or optical sensor device 9 is provided with electromagnetic inductioncircuitry.

The base station 2 may be powered through a power cord. The base station2 may alternatively be powered by a rechargeable battery which ischarged from time to time with a battery charger powered by the powersupply grid. The base station 2 has a receiving slot for physicallysupporting and storing the tooth brush when it is not used by a user.

The base station 2 and or separate optical sensor device 9 includes adata exchange circuit, for communicating data with a network 3, forexample the internet. Data may be transferred using aradio-communication link 31, as illustrated in FIG. 1 , with the antenna5 of the base station 2 and with the antenna 5 of a dedicatedcommunication equipment 8 or POA, connected to the network 3. In otherembodiments, transfer of data between the base station 2 and the network3 are performed through a wired link, for example ADSL.

The antenna 5 and transceiver means of the oral hygiene device 1 and/orcamera/optical sensing device 9 is also compatible with radiocommunication means of a mobile device 30 over a radio link 31. Theradio-communication link 31 is for example WiFi or GSM or Bluetooth ortheir derivatives or other suitable protocols. In some embodiments,radio links 31 are short range, local, radio communication links or aradio link 35 such as the ones used in cellular or other mobile phonesystems (GSM and derivatives for example).

The mobile device 30 is also able, via its radio communication circuits,to exchange data on a radio link 31 through the dedicated communicationequipment 8 or POA, on the network 3. In addition, or alternatively, themobile device 30 is able to exchange data on a radio link 35 directly onthe network 3.

A server 4 is connected to the network 3 by any suitable means. Server 4is defined broadly to include computing devices capable of storing andcomputational operations for example on the “cloud” in a computingnetwork. The server 4 may include storage devices, for instance memory,hard disk drives, flash memory, or other storage devices and includescomputational means under the control of a program. For the transfer ofdata, the oral hygiene device controlling circuit uses a predeterminedserver 4 address of the network 3. This predetermined address may bestored initially in the oral hygiene device 1 and/or updated laterthrough the network 3. The transfer of data between the oral hygienedevice 1 and server 4 may be performed: a) each time the oral hygienedevice 1 is replaced in the base station 2 in a batch configuration, b)at the direction of the user or the server 4, for example by user actioninitiating the transfer using the interface of the mobile device 30 or aweb page accessing the server 4 or c) in real time when oral hygienedevice 1 activities are detected, or d) the oral hygiene device 1 isremoved from the base station 2 or e) at other suitable intervals.

System Circuit Design and Network Architecture

As illustrated in FIG. 2 , the oral hygiene device 1 may include apressure sensor 10 and at least one motion sensor 11. The pressuresensor 10 detects force applied on the brushing side of the oral hygienedevice 1 when a user applies the bristles to their teeth. A motionsensor 11 may be provided for detecting motion on any or all three ofthe orthogonal axes of the oral hygiene device 1, or a motion sensor maybe able to detect accelerations or other motion characteristics in allthree axes. The signals output by the sensors are processed by a signalconditioning circuits 12. Examples of signal conditioning include:frequency and noise filtering, amplification, conversion, digital signalprocessing, and other techniques to optimize the detected signals foranalysis.

On other embodiments, the oral hygiene device 1 may not include anyelectronics and may be a standard toothbrush. In those embodiments, aseparate optical sensor 9 may perform the tasks of tracking the motionof the oral hygiene device 1.

The processed signals or raw data from the sensors are then stored inmemory 14 as determined by a control system 13 which may be a digitalsignal processor, microcontroller, or other processing component andwhich operations are controlled by a program 15. The memory 14 may beincluded in the oral hygiene device 1 or on a server 4 or othercomponent of the system 100. A program 15 may be updated through an oralhygiene device 1 interfacing circuit 16, a modem for radiocommunication, and its antenna 5 (and/or connector in case ofcontact/wired interface) or other interfaces of the oral hygiene device1. More generally, the oral hygiene device interfacing circuit 16 allowsinformation exchanges between the oral hygiene device 1, the opticalsensor device 9, and the base station 2 when the radio link 31 isestablished (and/or connectors of the tooth brush and of the basestation are mated together). The oral hygiene device 1 may contain apower supply circuit for powering the sensors and the circuits of theoral hygiene device 1 and it can include a rechargeable electric source17.

The base station 2 may include a base station interfacing circuit 20, amodem for radio communication, with an antenna 5 (and/or connector) toexchange information over link 31. In addition, the base stationinterfacing circuit 20 is able to establish a radio link 31 with thededicated communication equipment 8, for communication with the network3. The base station 2 may utilize a power supply converter 22 which isregulated 21 to provide appropriate voltage and current to the basestation circuits. Electrical connections (not illustrated) for providingcharging current to the oral hygiene device 1 from the base station 2may be provided. In some embodiments, the base station 2 may include arecharging circuit for recharging a battery or power supply of thetoothbrush, through inductive charging or a direct electricalconnection.

The base station 2, optical sensing device 9, or other separateelectronic device may also include a magnetic field transmitter 110 thatemits a magnetic field that may be sensed by an associated magnetometeror other magnetic field sensor 11. The magnetic field transmitter 110may be provided by utilizing the charging circuits or other circuitsthat already exist in the base station 2 or other electronic device. Forexample, the base station 2 may have a recharging coil that could alsoserve as a magnetic field transmitter 110. The recharging coil may befixed and in a known orientation, so as to create a magnetic field ofknown strength and polarity orientation. In some embodiments, the basestation 2 may include a recharging coil that generates a magnetic fieldwith a polar axis situated in a horizontal or vertical plane. In someembodiments, this may be a single axis magnetic field transmitter 110,such as in the case of a single axis recharging coil. In otherembodiments, 2 or 3 axis magnetic field transmitters 110 may beincorporated into the base station 2. This will advantageously allow fora fixed magnetic field(s) of known orientation so that a magnetometer 11on the oral hygiene device 1 may sense the strength and polarity of themagnetic field(s) in order to provide information regarding the positionand orientation of the oral hygiene device 1, or the relative changes inposition and orientation.

In some embodiments, the base station 2 or other electronic deviceseparate from the oral hygiene device 1 may also include a camera 9 thatmay detect visual patterns on the oral hygiene device 1. The camera 9may be any suitable camera that may detect a visual pattern on the oralhygiene device 1. For instance, the cameras provided with mobile phoneswould be suitable. In other embodiments, a standalone camera or opticalsensing device 9, a separate camera stand for a mobile phone, aconnected mirror or other camera or imaging device may be utilized.

In some embodiments, the base station 2 is passive and its circuits areunder the control of the controller 13 of the oral hygiene device 1 whenthey are communicating together, specifically when the link 31 is of thewired/contact type with connectors. In the embodiment represented onFIG. 2 , the base station has a control system 19 which controls itsoperations.

The dedicated communication equipment 8 may include a radio modemcircuit 27 and the appropriate electronics for communicating withnetwork 3. The dedicated communication equipment 8, is able to establisha radio link 31 with the base station 2 and/or a radio link 31 with themobile device.

The mobile device 30 includes at least a radio modem 26 for establishinga radio link 31. The operations of the mobile device 30 are under thecontrol of a control system 25, for instance, a central processing unitor μC, and of a program 15. The mobile device 30 includes an outputmeans such as a display screen and an input means such as a virtual ormaterial keyboard. Preferably, the input and output means of the mobiledevice 30 are used in the system to input information and to displayinformation, notably the results of computations performed by a server.The mobile device 30 may also include a camera 9 that is capable ofdetecting visual patterns supplied on the oral hygiene device fordetection of movement.

The program of the computational means of the server 4 allows storage ofsignals received from the oral hygiene device 1. Additionally, theserver 4 may analyze the data from the sensors to produce feedback andmotivational data regarding the user's performance in brushing theirteeth. These results may be accessible to the user on an internet pagehosted by the server 4 or transferred to another webserver for hosting.In a different embodiment, the previous operations and computations aredone fully or partially in the mobile device 30, the server 4 being usedfor general monitoring.

It should initially be understood that the disclosure herein may beimplemented with any type of hardware and/or software, and may be apre-programmed general purpose computing device. For example, the systemmay be implemented using a server, a personal computer, a portablecomputer, a thin client, or any suitable device or devices. Thedisclosure and/or components thereof may be a single device at a singlelocation, or multiple devices at a single, or multiple, locations thatare connected together using any appropriate communication protocolsover any communication medium such as electric cable, fiber optic cable,or in a wireless manner.

It should also be noted that the disclosure is illustrated and discussedherein as having a plurality of modules which perform particularfunctions. It should be understood that these modules are merelyschematically illustrated based on their function for clarity purposesonly, and do not necessary represent specific hardware or software. Inthis regard, these modules may be hardware and/or software implementedto substantially perform the particular functions discussed. Moreover,the modules may be combined together within the disclosure, or dividedinto additional modules based on the particular function desired. Thus,the disclosure should not be construed to limit the present invention,but merely be understood to illustrate one example implementationthereof.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someimplementations, a server transmits data (e.g., an HTML page) to aclient device (e.g., for purposes of displaying data to and receivinguser input from a user interacting with the client device). Datagenerated at the client device (e.g., a result of the user interaction)can be received from the client device at the server.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back endcomponent, e.g., as a data server, or that includes a middlewarecomponent, e.g., an application server, or that includes a front endcomponent, e.g., a client computer having a graphical user interface ora Web browser through which a user can interact with an implementationof the subject matter described in this specification, or anycombination of one or more such back end, middleware, or front endcomponents. The components of the system can be interconnected by anyform or medium of digital data communication, e.g., a communicationnetwork. Examples of communication networks include a local area network(“LAN”) and a wide area network (“WAN”), an inter-network (e.g., theInternet), and peer-to-peer networks (e.g., ad hoc peer to-peernetworks).

Implementations of the subject matter and the operations described inthis specification can be implemented in digital electronic circuitry,or in computer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Implementations of the subjectmatter described in this specification can be implemented as one or morecomputer programs, i.e., one or more modules of computer programinstructions, encoded on computer storage medium for execution by, or tocontrol the operation of, data processing apparatus. Alternatively, orin addition, the program instructions can be encoded on an artificiallygenerated propagated signal, e.g., a machine-generated electrical,optical, or electromagnetic signal that is generated to encodeinformation for transmission to suitable receiver apparatus forexecution by a data processing apparatus. A computer storage medium canbe, or be included in, a computer-readable storage device, acomputer-readable storage substrate, a random or serial access memoryarray or device, or a combination of one or more of them. Moreover,while a computer storage medium is not a propagated signal, a computerstorage medium can be a source or destination of computer programinstructions encoded in an artificially generated propagated signal. Thecomputer storage medium can also be, or be included in, one or moreseparate physical components or media (e.g., multiple CDs, disks, orother storage devices).

The operations described in this specification can be implemented asoperations performed by a “data processing apparatus” on data stored onone or more computer-readable storage devices or received from othersources.

The term “data processing apparatus” encompasses all kinds of apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, a system on a chip, or multipleones, or combinations, of the foregoing The apparatus can includespecial purpose logic circuitry, e.g., an FPGA (field programmable gatearray) or an ASIC (application specific integrated circuit). Theapparatus can also include, in addition to hardware, code that createsan execution environment for the computer program in question, e.g.,code that constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, a cross-platform runtimeenvironment, a virtual machine, or a combination of one or more of them.The apparatus and execution environment can realize various differentcomputing model infrastructures, such as web services, distributedcomputing and grid computing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astandalone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data, e.g., magnetic, magneto optical disks, or optical disks.However, a computer need not have such devices. Moreover, a computer canbe embedded in another device, e.g., a mobile telephone, a personaldigital assistant (PDA), a mobile audio or video player, a game console,a Global Positioning System (GPS) receiver, or a portable storage device(e.g., a universal serial bus (USB) flash drive), to name just a few.Devices suitable for storing computer program instructions and datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto optical disks; and CD ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

Oral Hygiene Device Design

As illustrated in FIGS. 3A-3C, the oral hygiene device oral hygienedevice 1 may include a handle 40, and a head 42 that may be removablyconnectable to the handle 40. The handle 40 may contain a motor that ismechanically connected to the head 42 and when activated vibrates ormoves the head 42 in manner that brushes a user's teeth when placedinside the mouth. The handle 40 includes a head interface 46 thatremovably attaches various heads 42 to the handle 40. The head interface40 contains leads 48 for both data and power transfer to various heads42. For example, certain heads 42 may include sensors that require powerand data transfer, and therefore power can be routed from the handle's40 power source to the head 42 through leads 48 that form a connectionwith the head 42 at the head interface 46. The may be various numbers ofleads 48 that form the connection on the head interface 46, for instancethere may be two leads 48 for power, and two leads 48 for data, threeleads 48 for power, three leads 48 for data, and other various numbersof leads. In some embodiments the head interface 46 will form awatertight seal with the head 42 to prevent water from entering theinterface and interfering with the electrical leads 48 power and datatransfer.

In some embodiments, the majority of the circuitry and costly componentscan be contained inside the handle 40 as opposed to the head 42, whichmay be disposable after a certain number of uses. This will minimize thecost of the replacement heads 42. For example, in some embodiments, thebattery, controller 13 may be contained in the handle 40, and any sensorprobes and circuitry to connect the sensor probes may be contained inthe head 42. In other embodiments, the head 42 may contain no circuitryor electrical components and will only provide a mechanical brushingfunction by supporting the bristles.

For instance, as illustrated in FIGS. 8A-8C, the electronics and batterymay be contained inside an insert 800 that is easily slidable into aconventional oral hygiene device 1. For instance, an oral hygiene device1 may include a chamber and connector that is connectable to a base andinsert 800 that slide into the chamber and the base forms a watertightseal with the connector. The insert 800 could be any manner of shapes(cylindrical, rectangular or others) that would slide inside a space ofthe toothbrush. In some examples, the base and connector will contain ascrew and thread mechanism to attach the toothbrush. In some examples,the connector and base will include a press-fit mating configuration foreasy connection and detachment. For instance, the connection may be madewith opposing wings on the connector and the base as illustrated inFIGS. 8A-8C.

In some embodiments, oral hygiene device 1 may only be a standardtoothbrush, or other standard oral hygiene device 1 that is commerciallyavailable and may not have electronics, or may only have electronics formoving the head to facilitate brushing. In some embodiments, the oralhygiene device 1 may only include patterns 120 or an attachment 130 witha pattern 120, and may not include any motion sensing electronics, ormay not include any electronics at all. Accordingly, in theseembodiments, the visual tracking software may be utilized to determineposition and orientation of the oral hygiene device 1.

The oral hygiene device 1 may also include a speaker 50 and variousvisual indicators 52 to provide audio and visual feedback to the user.For example, the handle 40 may contain a speaker 50 for playing music,substantive feedback, motivational phrases, remaining time elapsed,recommendations on brushing pressure, on whether certain quadrants havenot been adequately brushed, an announcement for completion of brushing,etc. Additionally, the oral hygiene device 1 may contain any number ofvisual indicators 52, for providing substantive feedback on the brushingincluding time elapsed, a LED indicator for when brushing is complete,warning indicators for brushing inappropriately, including indicatorsfor whether each quadrant has been addressed. In other embodiments, theoral hygiene device 1 may also utilize osteophony to convey audiomessages to the user.

As illustrated in FIG. 6 , the oral hygiene device 1 may contain ahandle 40 and head 42, where either or both may include a pattern 120for visual detection of movement and orientation by an associated camera9. For instance, in some embodiments, the back of the head 42 maycontain a pattern (i.e. “AB” with a circle and line as illustrated). Inother embodiments, the pattern 120 may be contained on an attachment 130that may be attachable to the head or on the neck, painted in thebristles, or other positions.

The handle 40 may also include a pattern 120, or in some embodiments maybe the only component that includes a pattern 120. The pattern 120 onthe handle 40 may be applied directly to the handle 40 or may be in anattachment 130 that clips or connects to the end of the handle 40. Thepattern 120 may be positioned at a convenient location on the attachment130 or on the handle 40 so that it may be detected in all angles ofnormal brushing activity. In some embodiments, the handle 40 may includemultiple patterns 120 on different sides for detecting differentorientations. For instance, in some embodiments, the attachment 130 maybe square or circular and have a different pattern 120 on each side inorder for the system to detect the orientation of the oral hygienedevice with respect to the camera.

The attachment 130 may be weighted so that the oral hygiene device 1stands by itself when set on a flat surface. For instance, a weight thatis heavy enough 130 to keep the oral hygiene device 1 upright may beapplied to the bottom of the attachment 130. In some embodiments, thismay be particularly useful if the attachment 130 is spherical on thebottom. This will give the oral hygiene device an entertaining qualitythat will be intriguing to children and even adults.

Pattern 120 may be applied using paint, other marking processes, or itmay use reflective coatings, mirrors, or fluorescent coatings. In someembodiments, pattern 120 may utilize color, or it may be grayscale.

Oral Hygiene Device without Pattern or Electronics

A standard oral hygiene device 1 or oral hygiene device may be utilizedwithout any electronics or patterns. As indicated, in some embodimentsthe position and motion of the oral hygiene device 1 will be detected.

Sensors

The oral hygiene device 1 or separate electronic devices (e.g. opticalsensors) may incorporate various sensors that detect certain attributesof brushing that may be analyzed to provide various feedback and othermotivational information to the user. For instance, one or more opticalsensors 9 may also be utilized on a separate electronic device to detectan orientation and movement of the oral hygiene device 1. For instance,the optical sensors 9 may be utilized to capture images of an oralhygiene device 1, and the images may be sent for processing to identifyits borders, shape, longitudinal axis, and orientation (for example byidentifying its bristles) In some embodiments, the optical sensor(s) 9may be utilized may detect patterns on the oral hygiene device 1 ratherthan the oral hygiene device 1 itself. The optical sensor(s) 9 utilizedfor pattern detection may be oriented in a direction to provide a visualline of sight to the pattern 120 on the oral hygiene device 1 that maybe on the head 42, handle 40 or on an attachment 130.

As another example, the oral hygiene device 1 may incorporate variousmotion sensors 11 to determine the quality of the brushing with respectto certain quadrants of the mouth or even individual teeth. The motionsensors 11 may include gyroscopes, accelerometers, magnetometers,gyrometers, and other various sensors capable of detecting positions,movement, and acceleration. These various motion sensors 11 may beincorporated either in the handle 40 or the head 42. However, it may beadvantageous to put the motion sensor 11 in the handle 40 because insome embodiments where the motion sensor 11 is in the head 42, themotion sensor 11 can experience at lot of additional motion (e.g., dueto brushing and engagement with teeth) that may interfere with detectinga position. In some embodiments, a magnetometer will sense a vector(s)of the earth's magnetic field. In some embodiments, a three-axismagnetometer will be used and in others a two or one axis magnetometerwill be utilized.

A magnetic field generator 110 may al so be utilized to generate a knownmagnetic field with a known polarity that may be sensed by amagnetometer incorporated into the oral hygiene device 1. The magneticfield transmitter 110 may be placed inside the base station 2 whichwould already have a recharging coil and/or interfacing circuit 20 thatmay be utilized to produce a detectable magnetic field. In otherembodiments, the magnetic field transmitter 110 may be a separateelectronic component in the base station 2 or in a separate physicalcomponent entirely. In some embodiments, the magnetic field transmitter110 would be in a stationary unit with a known orientation.

The oral hygiene device 1 may also incorporate various proximity sensorsthat detect the proximity of the oral hygiene device 1 to the mouth of auser. These may be incorporated at the head 42 or in the handle 40. Theproximity sensors may be utilized to acquire additional positionalinformation relevant to determining the brushing quality of the user.

Additionally, the oral hygiene device 1 may contain a pressure sensor 10to determine whether the user is applying appropriate pressure inbrushing their teeth. The pressure sensor 10 may be incorporated intothe head 42 which may be more easily flexible or utilize simple pressuretransducers or other components capable of measuring pressure.

In certain examples, the oral hygiene device may contain a pH sensor 10.The pH sensor 10 may be utilized to determine the salivary pH of a user.For instance, in some examples, a user may be instructed to place theoral hygiene device 1 in the user's mouth prior to using toothpaste ormouthwash, to test the salivary pH. Salivary pH may be indicative ofperiodontal disease or gingivitis.

In other examples, the oral hygiene device 1 or system 100 may alsoinclude a depth perception sensor. For instance, in some examples thedepth perception sensor will project a laser light grid or other laserpattern from the base 2, for example in place of camera 9, and include adetector that will detect and analyze distortions in the pattern todetermine depth. The depth perception sensor may be utilized todetermine more accurately the outlines of separate objects foridentification and motion tracking. For instance, the system 100 may beable to more easily identify the user's head and facial features, anddistinguish from the oral hygiene device 1. Accordingly, the depthperception device may be utilized to determine movement in a planeparallel to the line connecting the base 2 to the user.

In some examples, the oral hygiene device 1 may also contain a depthperception device. For instance, in some examples, the oral hygienedevice 1 may include a depth perception projector and sensor thatprojects onto the user's teeth. This may be utilized to form a map ofthe user's teeth and to detect holes or cavities in the user's mouth.

In some embodiments, various heads 42 may incorporate a camera 56 thatwill detect various aspects of tooth quality that may or may not berelated to brushing quality. For example, a camera 56 including a nearinfrared camera 56 may be able to be utilized on an oral hygiene device1 to collect data indicative of demineralization or dental caries ordental decay. For example, the oral hygiene device 1 may utilize certainwavelengths that are particularly suited to detect these abnormalities,for instance in the 1300-1400 nm range. In some embodiments, the oralhygiene device 1 may also contain a light source 58 that will be focusedtowards the teeth during brushing and can be utilized by the camera todetect certain abnormalities.

In some examples, the output of the camera 56 may be utilized by thesystem 100 to form a tarter map of the user's mouth. For instance, insome examples, the system 100 may utilize the images from the camera 56to identify tarter based on the reflection wavelengths, and build aschematic or other representation of the tarter for the user's mouth.For instance, the camera 56 may utilize certain wavelengths and/orfilters that reflect tarter more readily, and identify which areas ofthe mouth have more or less areas of tarter, or other issues. Forinstance, tarter may cause light to reflect less, and may also reflectdifferent wavelengths. These different patterns can be filtered ordetected by machine learning algorithms by the system as described infurther detail below.

Certain cameras 56 and potentially light sources 58 may also beimplemented to detect levels of plaque on the teeth and changes inlevels of plaque during brushing. In certain embodiments, infrared ornear-infrared light sources 58 and an appropriate camera 56 that detectsand records light in this wavelength range may potentially allow for thedetection of plaques.

Program for Brushing Analysis and Feedback

The systems various sensors and optical sensors may gather data relevantto the quality of brushing by a user or the overall dental health of auser's teeth. This data may then be processed using programs orapplications installed in various portions of the oral hygiene devicemonitoring system 100. Accordingly, as described above, data from thesensors and optical sensors may be processed by a program executed bythe oral hygiene device's 1 control system 13 or alternatively aprocessor on the mobile device 30, another associated computing device,or the server's 4. The system's 100 processing and analysis of the datawill result in output data representing feedback relevant to a user'squality of brushing. This feedback may be communicated through audiofeedback through the oral hygiene device 1 speaker 50, visually on theoral hygiene device 1 indicators 52, or both on an associated mobiledevice 30 or when accessed on a website hosted or in communication withthe server 4.

FIG. 4 illustrates an example of the process by which the oral hygienedevice 1 evaluates the brushing quality of a specific user. Each thesteps may not be required in a specific embodiment, but variouscombinations of these steps may be implemented in an oral hygiene devicemonitoring system 100. First, the user may input a user profile 70 intothe oral hygiene device monitoring system 100, to allow the user tocalibrate the oral hygiene device 72. In some embodiments, the oralhygiene device 1 may be calibrated at the factory, by the user or both.After calibration, the user may pick up the oral hygiene device 1 andbegin brushing her teeth. The oral hygiene device or associatedmonitoring devices (e.g. camera) would then determine that brushing isinitiated 74, and start recording sensor data 76 during brushing. Thenthe oral hygiene device monitoring system 100 may analyze the sensordata 78 to output brushing quality feedback 80 to the user.

User Profile

Accordingly, a user profile 60 may be entered 70 for each particularuser of a certain oral hygiene device 1 or associated with a specificaccount stored on the server 4, inside the oral hygiene device, basestation or other computing devices. Upon initialization for of a neworal hygiene device or new account on the server 4, a user may entertheir information that may be utilized to determine optimal brushingtimes and characteristics. For example, a program 15 may first request auser's name, gender, height, weight, age, and certain questions on theirdental history. The user profile 60 may then be associated with certaindata recorded during use of the oral hygiene device by the user,including calibration data that is specific to certain oral hygienedevices 1, associated optical sensors 9 or is generic and can be appliedto any oral hygiene device 1 or optical sensors 9 connected to thesystem 100. In some embodiments, a user may upload a picture ofthemselves, or a program may be initiated that uses the opticalsensor(s) 9 to capture a picture of the user from a distance from whicha user would typically brush their teeth.

Detecting Usage

A tooth brushing monitoring system 100 may also determine whether usagehas taken place and the number of usages per day. In some embodiments,the oral hygiene device 1 and/or optical sensor detection system detectsmotion data through motion sensors 11 and/or optical sensors 9 andanalyzes the data to determine whether usage has occurred, or whetherthe brush has been moved or usage is feigned.

When motion indicative of usage is detected, the oral hygiene device 1or optical sensor device 9 may store the positional and motion data inits memory 14 for later analysis. For example, this will prevent therecording of false positives, for example when a user moves the brush ina medicine cabinet, or from children circumventing the system by brieflyshaking the toothbrush.

For example, movement indicating usage may be associated with a certainacceleration level and/or frequency that is characteristic of aparticular user. In other embodiments, a user may push a button orswitch on the oral hygiene device 1, base station 2, or device with anoptical sensor 9 to wake up the sensors on the device, which will thenbegin recording data. Accordingly, the system will determine whenbrushing is initiated 74. In some embodiments, this will be performedautomatically, for example, upon the detection of certain accelerationsand frequencies. Accordingly, once the user picks up the toothbrush, themotion sensors 11 may begin recording the data 76 and sending it to anyof the various control systems 13 in the system 100 to analyze it 78 forcharacteristics associated with brushing.

For instance, the oral hygiene device 1 will generally be resting on itsbase 2 pointing upwards prior to use, in an orientation that would notbe suitable for brushing by a user holding the oral hygiene device 1.Accordingly, once the user picks up the oral hygiene device 1, the oralhygiene device 1 will generally be rotated roughly 45 degrees to be heldprimarily horizontal during brushing. Accordingly, one thresholdcriteria for determining that brushing is initiated 74 would be whetherthe oral hygiene device 1 has been titled within a certain angle rangeindicating the oral hygiene device 1 is horizontal or near horizontal.This could be an angle range of 20 degrees, 5 degrees, 10 degrees, orother suitable ranges. Additionally, a series of calibration sessionsmay indicate a suitable range. Of course, this could be detected byoptical sensors 9 and/or motion sensors 11.

In some embodiments, the user may turn on the device and the opticalsensor 9 (and/or depth sensor) may begin recording. Then the system maylook to determine when the pattern 120 is at a certain height indicatingit is close to the user's mouth. This may be combined with accelerationinformation detected by the optical sensor(s) 9 and analyzed as abovefor the motion sensors 11.

In other embodiments, the determination of whether brushing hasinitiated 74 and whether or not it has ceased may be performed by thesystem 100 using a statistical analysis of the motion data from motionsensors 11 and/or optical sensors 9. This statistical analysis may beperformed by correlating data from the motion sensors 11 and/or opticalsensors 9 to previous tooth brushing or calibration data, or data storedfrom other users. For example, after performing the analysis, a certainthreshold of correlation of the motion data with previously recordedcalibration data that is associated with usage may be determined thatindicates brushing has initiated 74 or is in progress. Accordingly, oncethe user begins brushing, the system 100 may record that usage has beeninitiated 74 and record the data 76 in memory 14 until usage stops asbrushing data, for instance after the correlation falls below a certainthreshold.

For instance, utilizing an optical sensor 9 system setup, the opticalsensor 9 may output data that includes images of the oral hygiene device(e.g. toothbrush or water flosser) and the user. The data may be sent tovarious control systems to be processed and analyzed for motion. Forinstance, the image processing algorithms may first determine a boundarycondition to identify the boundaries of the lips/mouth of the user, theteeth, the head, the oral hygiene device, the handle, head, bristles,water flosser, etc.

Identifying the boundaries of the human characteristics could beutilized using standard boundary identifying algorithms that generallymay utilize threshold changes in certain colors (light frequencies).Once the boundaries of the oral hygiene device 1 are identified, alongitudinal axis could be identified, and potentially an orientation ofthe bristles (if it is a toothbrush) to determine an angular orientationabout the longitudinal axis. This will allow the system to determine thegeneral orientation and motion of the toothbrush, with time stampedframes from the imaging device.

Then, the toothbrush can be identified as on a certain side of the mouthby analysis of the relative positions of the toothbrush and features ofthe mouth. Furthermore, the orientation of the toothbrush with respectto the side of the mouth it is on can be utilized to determine whichsection or portion of a user's teeth are being brushed or water flossed.For instance, of the bottom of the upper molars are being brushed on theright side, then the visual system would determine the toothbrush is onthe right side of the mouth with the bristles facing up.

Furthermore, once the outline and orientation of the toothbrush 1 isdetermined in each frame, the time stamps of each frame can be utilizedto determine the motion of the toothbrush. For instance, the change inpositions, (time and distance) can calculate speed and acceleration ofchanges. Accordingly, as with motion sensors 11, the image data can beutilized determine the motion of the oral hygiene device. Accordingly,that motion may be utilized to determine compliance with brushing orother oral hygiene standards as disclosed further herein.

The analysis of motion data (processed from motion sensors 11 or imagedata from optical sensors 9) may utilize a fingerprint or signature typeanalysis that only compares relative movements. The signature may bedetermined based on the acceleration in certain axes (as detected bymotion sensors 11, time stamp image data, or other methods), as themotion of brushing teeth is generally performed in a relatively rapidmotion that is uncharacteristic of any other incidental movement of theoral hygiene device 1, for example, to put it back in the cabinet.Additionally, the frequency of the brushing may be monitored, asbrushing is generally a rapid periodic motion, and therefore variousbandpass frequency, low-pass, and Kalman filters may be used or othertechniques to identify certain frequencies of interest and amplitudes inthose frequencies that indicate brushing.

These amplitudes in frequencies may be certain frequencies that reach athreshold amplitude, that are associated or determined to indicate auser is brushing. For example, certain frequencies in horizontal orvertical axes may be required for the system 100 to determine brushingis initiated 74, or certain periodic accelerations that reach certainthresholds may be required for the system to determine brushing hasinitiated 74. In some embodiments, this may a frequency of 1-5 Hz. Oncethe data analyzed by the controller 13 falls below a certain thresholdthat indicates use, the system 100 may stop recording data or determinethat brushing has stopped.

In addition to statistical analysis, the system may detect movementindicating usage or actual brushing by using filtering and thresholdanalysis. For example, the system 100 may first filter the data from themotion sensors 11 to pass frequencies only in a certain band (asbrushing is periodic) and monitor those frequencies to detect when thefor one the signal in that reach a threshold for at least a certainnumber of cycles or duration to determine the user is brushing. Forexample, if a user brushes their teeth at an average of 1-5 Hz (orpotentially less in the case of a motorized toothbrush), a band passfilter of 1-5 Hz may be implemented.

Thus when the system 100 detects that amplitude of the frequency band inthe 1-5 Hz range reaches a threshold indicating use, the controller 13may begin to record data from the sensors in the memory 14 for theduration of time the motion data indicates the oral hygiene device 1 isbeing used. Additionally, periodic accelerations in certain axes orangular acceleration (for circular brushing) that reaches certainthreshold amplitudes may also be used to indicate brushing hasinitiated. The analysis of the data may also be affected by whether theoral hygiene device 1 includes an electronic motor to vibrate the headto assist in brushing. In those embodiments, the data may be filtered toeliminate the high frequency acceleration and other noise created by theelectronic motor.

Quality of Brushing—Movement Types

In some embodiments, the quality of brushing based of the type ofmovements the user performs using the oral hygiene device 1 may bedetermined. Dentists have indicated that certain movements are more orless beneficial for brushing. Different types of movements includecircular movements, in both clockwise and counterclockwise motions, tipto root motions, and left to right motions.

In some examples, the system 100 may determine whether the length of thebrushing stroke. This could be by any combination of the methodsdisclosed, including by determining a magnitude of acceleration and timeof acceleration in each direction for strokes. For instance, strokes maybe filtered out by identifying a regular pattern or filtering at certainfrequencies and magnitudes. For instance, acceleration at a certainamount in certain directions with respect to the toothbrush will likelyindicate brushing strokes.

Most brushing will take place in the plane of the bristles, because thestrokes will be optimized for contact with the tips of the bristles withthe teeth using brushing motion. Accordingly, the system may filter outacceleration in the plane of the bristles, or within a suitabletolerance, to further identify acceleration or movement that relates tobrushing strokes.

The system 100 may determine if these motions are being performed therelative amount of these motions by filtering the data from motionsensors 11 or optical sensors 9 in certain axes that is indicative ofeach motion. For example, the data from motions sensors 11 or opticalsensors 9 may be filtered in an axis horizontal to gravity, and thecontrol system 13 or other system 100 processors may process the data todetermine whether the acceleration, frequency, or other motion datareached a significant enough amplitude in a certain direction toindicate that particular motion is performed.

In the case of image data, in addition to detecting thresholds ofacceleration or velocity that indicate brushing, the optical system maydetect when the oral hygiene device 1 is within a certain proximity tothe user's mouth, or inside the user's mouth to determine brushing isinitiated. For instance, if the head can be identified, whenever thehead is inside a region defined as being inside the user's mouth thesystem can determine that the user is brushing his or her teeth quitereliably.

In other examples, acceleration alone may be utilized to determinewhether back and forth motions are being used, or circular motions. Inother embodiments, the acceleration data from motion sensors 11 may beintegrated to determine the actual movement of the oral hygiene device 1to evaluate the type of brush strokes utilized. The analysis of the datamay also be affected by whether the oral hygiene device 1 includes anelectronic motor to vibrate the head to assist in brushing. In thoseembodiments, the data may be filtered to eliminate the high frequencyacceleration and other noise created by the electronic motor.

In some embodiments, an electronic motor to vibrate the head 42 may beincluded in the oral hygiene device 1. In those embodiments, the motiondata recorded by the sensors relating to brushing movements would have asmaller amplitude than for a manual brush 1. This is because users ofmanual toothbrushes, without the assistance from the electronic motorand moving head 42, will brush their teeth with more vigorous motions.Accordingly, the algorithms utilized to analyze the motion data todetect, use, motion, and location of oral hygiene device will bemodified to account for the lower amplitudes and/or different motions,and include filtering of the high frequency noise from the motor.Accordingly, in some embodiments, the thresholds set for the amplituderequired to detect or indicate a brush stroke would be less, as a userusing an electronic oral hygiene device generally moves the brush at aslower pace, and makes more straight line movements.

Furthermore, pressure sensor 10 may also be utilized to determinewhether brushing is actually being performed, or in combination with themotion data from above. For instance, the user may be moving the oralhygiene device 1 around but not pressing on the teeth. Therefore,accordingly, requiring both motion of a certain frequency, amplitude, orfeatures, and a certain pressure will eliminate many false positivesfrom incidental movement or pressure of the brush that is not happeningduring brushing. Accordingly, the pressure sensor 10 may output a signalthat is analyzed 78 by controller 28 to determine a pressure beingapplied to the teeth. If the pressure is above a certain threshold, thesystem 100 may indicate or determine it is likely that a user isbrushing. This analysis may be performed by statistical analysis,threshold analysis or other suitable calculation methods for determininga likelihood of brushing based on the amount and/or sustained nature ofthe pressure recorded by pressure sensor 10.

In some examples, the system 100 and toothbrush 1 may develop a libraryof specific types of brushing strokes or motions, and give the userfeedback on the brushing strokes or motion the user implemented fortheir brushing. For instance, the system may retain a dictionary ofmotion types for tooth-brushing, and rank the motion types, and thequality of each motion type.

Examples of motion types may be the following:

Type Motion Algorithm Examples Quality Horizontal Brush along lineAcceleration in plane Not good: Scrub of dentition in of bristles and inone Causes horizontal axis switching Cervical strokes positive andnegative. Abrasion Bristles horizontal. Bristle axis facingperpendicular to gravity. Machine learning. Sulcular Place brush tip at45 Bristle axis facing Very good: Brushing degrees and place about 45degrees to removes tips of bristles in gravity. Acceleration plaquegingival sulcus. in plane of bristles below the Vibrate back and and inone axis gingival forth with very switching positive margin. smallstrokes. and negative. Very small movements. Machine learning. CircularMove brush in a Continuously Least circular motion. changingacceleration effective in the plane of the brushing bristles. Machinetechnique. learning.

Identification of Specific Users

The system 100 may include a stored user profile associated with theuser's stored tooth brushing (or other oral hygiene) data and thedemographic data of the user that includes the age, size and gender ofthe user. During or after the step of monitoring oral hygiene (e.g.tooth-brushing) activities, the oral hygiene device 1, mobile device 30,or server may automatically seek to match the user with at least oneuser profile using at least one predetermined rule or algorithmdepending on the user profile and of past data. If the user is not aregular user of the oral hygiene device, said user identifies him/her asa guest on the mobile device.

In a step of user identification, a specific user may be associated withthe oral hygiene device and presumed to be the user. If multiple usersfor a given oral hygiene device are utilized, to associate a user with abrushing activity at least in the oral hygiene device and possibly inthe mobile device and/or the server at least for reference purposes forthose last two.

In embodiments that utilize a camera, the user profile may have apicture of the user uploaded or associated with the profile. This willallow the visual based recognition system to automatically determine aspecific user associated with the profile. In some examples, the system100 may capture a picture with the camera, and identify the specificuser by comparing the picture taken initially with each user thatinitiates brushing. In some examples, the system may utilize theposition of the user on the frame of the camera to determine the user(e.g. by estimating the user's height or relative height). In otherexamples, the system 100 may utilize machine learning and computervision principles to match features of the user and determine whichsaved user (and associated user profile) is currently brushing. Forinstance, eye color or other facial recognition techniques may beutilized to match users on a reliable basis, especially if each unitonly has 2, 3 or 4 users total to distinguish.

In other examples, use data associated with a particular user may beutilized to identify that user. For instance, the usage data for aparticular user may be associated with a certain acceleration leveland/or frequency that is characteristic of that user. For instance, aparticular user may have a certain frequency range of tooth-brushing, ormay begin on a particular side of the mouth. In some examples,combinations of physical features, motion data, or other indicators maybe utilized to identify users and associate the user with their profiledata.

Usage Monitor for Replaceable Heads

Additionally, the total number of uses for a replaceable oral hygienedevice 1 head 42 may be detected and stored in the memory 13 of the oralhygiene device 1 or associated with the server 4, to monitor the usefullife of a single oral hygiene device 1 head 42. Thus, the system 100could provide a notification to the user, for example on their mobiledevice 30 or through the webpage interface to the server 4, once aparticular oral hygiene device 1 head 42 has been used a certain numberor duration of time. Once a head 42 has been exhausted, the user willdisconnect and reconnect a new head 42, which could be interpreted orrecorded by the system 100 as attaching a new oral hygiene device 1.Additionally, each head 42 may incorporate a unique identity module 62that stores an identifier for that specific head 42, so when the samehead 42 is reattached, for example, for cleaning or other purposes, thesystem 100 will not assume a new head 42 has been attached.

Brushing Time and Position

Next, the brushing data recorded above may be analyzed 78 to providefeedback on the positional and time quality of brushing 80. This may bepresented after brushing has completed or in real time to provideinstantaneous feedback to the user. For example, the motion sensors 11and/or optical sensors 9 may detect positional information of the oralhygiene device 1. Using the accelerometer and gyrometer data, visuallydetected data from the optical sensors 9 and patterns 120 or othermotion sensors 11, the position, orientation, and movement of the oralhygiene device 1 may be determined and extrapolated to calculate therelative positional movement of the head 42. For instance, if thedistance, orientation, and direction of the motion sensors 11 to thebristle portion of the head 42 are known, the system 11 will be able todetermine the position of the head 42, and its relative motion,orientation, and coverage area based on the relative motion of themotion sensors 11.

This may be calculated by initially calculating a reference coordinateone the oral hygiene device 1 detects motion, and recording the relativemovement with respect to the initial coordinate(s) to determine asignature of the brushing. This may be performed by calculating themovement of the head 42 by calculating the changes in orientation of thebrush and the movement due to acceleration recorded by the gyrometer oraccelerometer, or visually detected by the optical sensors 9. Thesecalculations may be performed by the control system 13, or otherprocessors that are utilized in the system 100, including a mobile phoneprocessor executing an application on the mobile phone, or aprocessor(s) on a server running software for analyzing the data.

Brushing Time and Position—Magnetometer

In some embodiments, a magnetometer sensor 11 may provide additionalpositional information by detecting the polarity, a directional vectorand strength of the earth's magnetic field and/or the magnetic fieldgenerated by a magnetic field transmitter 110. In some embodiments, thismay be a magnetic field transmitter 110 located in the base station 2,as the base station 2 is stationary, and the magnetic field transmitter110 would have a known polarity and direction (e.g., horizontal orvertical and in a certain direction). This may be detected by amagnetometer 11 located in an oral hygiene device 1. Accordingly, thestrength of the magnetic field detected by the magnetometer sensor 11will give an estimate of the distance the oral hygiene device 1 is fromthe base station 2 along a sphere or oval curve or egg shaped curverepresenting the magnetic field lines surrounding the base station 2 orother associated device.

Additionally, the magnetometer will be able to sense the vectordirection of the magnetic field along the magnetic field lines. In otherembodiments, the distance of the oral hygiene device 1 from the basestation 2 may be calculated using: the magnetic field orientationdetected by the magnetometer sensor 11 emitted from the magnetic fieldtransmitter 110 together with the attitude information of theaccelerometer and gyrometer. This is possible utilizing the knowledge ofthe shape of the magnetic field and its vectors at particular distancesfrom the transmitter and its polarity. For example, the inclination ofthe magnetic field with respect to gravity will change predictably atvarious distances from the magnetic field transmitter for example.Additionally, by comparing the gravity vector with the vector of themagnetic field, the angle may be utilized to calculate the height alongthe shell or magnetic field lines. This is because the angle of thefield with respect to gravity will be different at each height becausethe egg shell will be directed at a different angle with respect togravity at each height. This will be especially true when the north andsouth poles are oriented in a vertical direction, so that the magneticfield lines have components that change from horizontal near the bottomand top of the egg shells to vertical at about the height of thetransmitter.

Accordingly, the magnetometer data combined with the accelerometer dataand/or optical data may be utilized to determine position of an oralhygiene device 1 within a ring that is situated in a horizontal planethat is of equal magnetic field strength around the transmitter 110. Forexample, FIG. 5 illustrates an embodiment of a base 2, with a singlecoil magnetic field transmitter 110 that generates shells that haveslices with B (magnetic) field vectors that change direction in avertical plane. Sensors 11 on the oral hygiene device 1 may include amagnetic field sensor 11 to detect the magnetic field generated by themagnetic field transmitters 110 and an accelerometer 11 to detect, amongother accelerations, the acceleration of gravity a_(g). Accordingly, themagnetometer 11 on the oral hygiene device 1 may be positioned with theB field generated by the transmitter 110. Accordingly, a magnetic fieldstrength and direction (B_(t)) may be determined by the magnetometer 11.Simultaneously, an accelerometer 11 may detect the acceleration fromgravity as vector a_(g). The system 100 may the analyze that data, anddetermine an Q° between the gravity vector a_(g) and magnetic fieldvector B_(t).

The angle Q° may then be able to be used to determine a ring R whereinthe oral hygiene device 1 position is known to be at some point alongring R. For instance, first, the strength of the magnetic field Bdetected by the magnetometer 11 can be utilized to determine whichmagnetic field egg or donut shaped shell the oral hygiene device is on(e.g., B₁, B₂, or B₃). This would narrow the position of the oralhygiene device 1 down to a hollow shell or donut surround thetransmitter 110. Next, Q° can be unitized to calculate a verticalposition on the shell or donut and therefore narrow the position down aring R surrounding the transmitter 110. Accordingly, if the system 100detects movement to a new ring R, the system will have information aboutthe possible direction and distances travelled. This information will beenhanced by the accelerometer data, which will provide furtherinformation about the direction travelled. This, for example, mayprovide certain upper or lower bounds of direction and/or distancetravelled by the oral hygiene device 1. Thus, this system 100 and dataprocessing technique may be utilized to provide relative position andmovement information of the oral hygiene device 1 and applied todetermine tooth brushing position in the mouth as described herein.

In some embodiments, a magnetometer sensor 11 may sense a magnetic fieldtransmitted by a magnetic field transmitter 110. In order to do this,this magnetic field data must be filtered out from extraneous noise andespecially from the earth's magnetic field. The earth's magnetic fieldwill be of relatively constant strength and direction at a givenlocation on the scale of movement of a toothbrush. Additionally, theearth's magnetic field is quite large and most magnetometers arecalibrated to the scale of the earth's magnetic field. Accordingly, touse a magnetometer calibrated for the earth's magnetic field, forinstance, in embodiments that detect both the magnetic field of theearth and of the transmitters 110 to calculate orientation and positionchanges, certain filtering algorithms must be used to distinguish thetwo. In some embodiments, the critical data that may be utilized tofilter the two is that the strength of the earth's magnetic fieldremains relatively constant on the scale of oral hygiene devicemovement, compared to the change in magnetic field detected that istransmitted by the magnetic field transmitter 110. Therefore, temporalfilters may be applied to distinguish the two magnetic fields. In someembodiments, the magnetic field transmitted by transmitter 110 may bepulsed at a certain frequency or amplitude in order to be easilyfiltered out using signal processing and data analysis.

In some embodiments, the magnetic field directional component of thevector data output from the magnetometer 11 may also be utilized tocalculate a relative position change and/or orientation. For example, insome embodiments, the transmitter 110 may transmit a magnetic field witha vector orientation illustrated in FIG. 5 . Directional or vectorinformation detected by a magnetometer from a magnetic field B createdby transmitter 110 may be useful to provide orientation and or relativepositional information. This data may then be combined with data fromthe directional vector information detected from the earth's magneticfield which would likely have an orientation that differs from theorientation of the earth's magnetic field. Accordingly, the magnetometer11 may detect vectors indicating the direction of the earth's magneticfield and the vectors of the magnetic field of the transmitter 110 toprovide a reference for orientation in space and for changes inposition.

In one example, the magnetometer sensing the earth's magnetic fieldcould sense an orientation of the oral hygiene device 1 in a horizontalplane as would a compass or about the Z axis illustrated in FIG. 5 . Insome embodiments, this may include a vector not only in the horizontaldirection but in the vertical direction that will be constant and basedon the inclination of the earth's magnetic field at a particulargeographic location. In other embodiments the transmitter 110 may beplaced in a diagonal, horizontal, vertical or other known orientationwhich is used as a reference point to calculate orientation of themagnetometer 11 and positional changes of the magnetometer 11 and oralhygiene device 1.

Furthermore, calibration may be necessary based on the relativeorientation of the oral hygiene device 1 when being used by a user andthe base station's magnetic field transmitter 110. As the shape andorientation of magnetic field lines change based on the position withrespect to the magnetic field generator, different formulas,calibrations, or adjustments may need to be performed. For example, nearthe side of a magnetic field generator, the flux lines may be orientedin a first direction; however, near either polar end of the transmitter110 (e.g., above or below, or the right or left side) the flux lines maybe oriented perpendicular to the first direction near the side or middleof the transmitter 110. Additionally, the user may be required to brushher teeth a certain distance from the transmitter 110 to ensure a propermagnetic field for measuring is generated in that position. Thebeneficial aspect of the coils being physically oriented in the verticaldirection is that the magnetic field generated is relatively the same onall sides of the transmitter 110 (and by proxy, the base station 2)assuming relatively the same height, and therefore calibration mayrequire simpler algorithms and/or processes. However, if the coils liein a horizontal plane, depending on the orientation of the base station2, the magnetic flux lines may be traveling in different directions. Inthat embodiment, the user may be instructed to swivel the base station 2to point it in a certain direction with respect to the user where theuser typically brushes their teeth. In other embodiments, thecalibration may be able to utilize or accommodate for changes in thevector direction of the magnetic field transmitted by the transmitter110.

The data from a magnetometer 11 may be combined with the data outputfrom an accelerometer 11 or data from the gyrometer 11 and accelerometer11 or optical sensor 9 to determine orientation and/or position. Forexample, the accelerometer 11 and/or optical sensor 9 data may first beutilized to determine the orientation of the oral hygiene device 1 withrespect to gravity during periods of relatively little acceleration, sothe acceleration of gravity would be detected by a three axisaccelerometer or image analysis so it could be determined whether theoral hygiene device 1 is tilted about a lateral axis that is in ahorizontal plane, or how it is rotated about a its longitudinal axis (alongitudinal axis running lengthwise through the toothbrush) withrespect to a gravity pointing down towards the earth.

If the accelerometer data indicates the oral hygiene device 1 isexperiencing relatively small amounts of acceleration relative to normaltooth brushing movements, the control system may determine that the oralhygiene device 1 is relatively still and the accelerometer data can beutilized to determine a vector oriented in the direction of gravity.

If the accelerometer data is combined with the gyrometer data,deviations of the attitude of the tooth brush with respect to gravitycan be determined to calculate the attitude of the toothbrush. Forexample, the initial recording of the gravity vector may then be used asa reference vector to determine attitude. Then deviations from thisinitial position or attitude determined by the accelerometer andgyrometer may be determined by calculating orientation changes from theinitial reference vector based on gyrometer data. Accordingly, theaccelerometer or accelerometer and gyrometer data may be utilized todetermine which way the bristles are facing. For example, in someembodiments, the accelerometer and gyrometer data may be analyzed todetermine whether the bristles are facing upward forwards or downwards,front, left and right side, potentially narrowing down to a specificquadrant or other division of the mouth. For example, when the oralhygiene device 1 bristles are facing down they could only be brushingthe tops of the bottom teeth. If the oral hygiene device 1 bristles arefacing up, it could only be brushing the crowns or tips of the upperteeth. As a further example, when the oral hygiene device 1 is facingright, it could only be brushing the left sides of the molars, etc.Additionally, in some embodiments, the orientations with respect togravity will be deterministic as a vector representing gravity alwayspoints in the direction towards the floor or earth, and brushing willvery likely take place standing up straight.

Therefore, it is possible to determine the orientation of the bristleswith respect to the teeth in some aspects or planes of orientation. Incertain embodiments, these calculations may be made deterministically,or estimated using statistics and/or a retrofitted model of the mouth.For example, if the accelerometer senses the toothbrush's bristles arefacing down within a certain margin of error or within a statisticallysignificant margin, then the system can determine that the surfacesbeing brushed are likely the tops of the lower teeth (or the tongue forexample). In other embodiments, as disclosed herein, the brushingorientations may be calculated relative to each other, so a model of themouth may be fit each time the user brushes their teeth.

To determine the orientation in a horizontal plane perpendicular togravity, the earth's magnetic field or the magnetic field generated bythe base station 2 transmitter 110 for example, may be utilized. In someembodiments, the magnetic field transmitter 110 will generate a magneticfield with directional vectors that change direction in a horizontalplane that is perpendicular to gravity. Accordingly, even if theabsolute direction is not known, a relative orientation of the oralhygiene device 1 with respect to the magnetic field detected by themagnetometer 11 and emitted by the transmitter 110 may be determined ina horizontal plane. Accordingly, relative changes of this orientation ina horizontal plane could be determined and a path of movement along areference coordinate system may be determined. In some embodiments, thereference coordinate system may be based on one of the initial or anyother data points recorded during a tooth brushing session. As disclosedherein, these relative changes may be utilized to calculate thepositions or reconstruct the shape by using relative positions after auser is finished brushing.

In some embodiments, data from the gyrometer, magnetometer, opticalsensor and accelerometer may be utilized to determine an orientationwith respect to gravity and either or both magnetic field of the earthand of the transmitters 110. In some embodiments, orientation may bedetermined and confirmed or validated by the different sensors 11 oroptical sensors 9. For example, the angular velocity from a gyrometermay be integrated to determine orientation changes in certaindirections, but errors from integration may be corrected using themagnetometer readings and/or optical sensor 9 data. In otherembodiments, either or both the magnetometer, gyrometer andmagnetometers may be utilized in various combinations to determineorientations.

In some embodiments, orientation alone may be utilized to determine theposition of the oral hygiene device 1 using statistical analysis asdisclosed herein. In other embodiments, the gyrometer and/oraccelerometer detected inertial changes may be used in conjunction withor separate from magnetometer 11 readings to be provide furtherindications of movement or distance to calculate an estimated positionof the oral hygiene device 1 bristles. In some embodiments, theorientation information may be combined with data output from sensors 11or optical sensors 9 that indicate translational inertia, or positionalchanges. Depending on the location of the inertial sensor, theorientation information may be combined with the inertial movement datato indicate the position of the bristles. For instance, if a motionsensor 11 is located inside the oral hygiene device 1, but in the middleof the toothbrush's longitudinal axis where the user would hold the oralhygiene device 1, changes in orientation would move the head withoutmoving the position (although it may rotate in place) of the motionsensor 11. Accordingly, the orientation information can also be used tocalculate a position in space of the bristles relative to the positionof sensor 11.

Movement or positional changes may also be calculated from data outputby the magnetometer 11. In some embodiments, the magnetometer 11 maydetect a magnetic field generated by a magnetic field transmitter 110 inthe base station 2 or another stationary component that is separate fromthe oral hygiene device 1. In this embodiment, the strength andorientation of the magnetic field sensed by a single-, two-, orthree-axis magnetometer 11 can provide additional data on the positionalmovement of the oral hygiene device 1 with respect to the magnetic fieldtransmitter 110 in the base station 2, which is fixed and stationary.Accordingly, the increase or decrease in magnetic field will indicatethat the oral hygiene device 1 is moving towards or away from the basestation 2. Additionally, changes in the orientation or inclination ofthe magnetic field and its polarity output by the magnetometer 11 can beutilized to determine translational and position. This information maybe utilized to determine, based on calibration or experimental data, howmuch a per unit change in magnetic field strength is equivalent to indistance based on the direction of movement (as different directionsthrough a magnetic field will have higher or lower rates of change). Inother embodiments, the per unit change in inclination or shape of themagnetic field may also be correlated to changes in position. Therespective orientation of a horizontal magnetic field may be differenteach time a oral hygiene device system as disclosed herein is set up,including such a system that incorporates a magnetometer in the basestation 2. This is because how each user stands as they are brushingtheir teeth, and/or how the base station 2 is oriented on the top of thecounter may change for each user and even change somewhat each time andafter cleaning the countertop, etc. Accordingly, adaptive statisticalanalysis may be utilized to determine relative positions of the bristlesof the oral hygiene device 1.

Positional data calculated from magnetic field data output from amagnetometer 11 may be combined with inertial data to calculatepositional changes in the oral hygiene device 1. In some embodiments,the inertial data may be utilized to calculate distance traveled and themagnetometer data may also be utilized to eliminate errors, and/orprovide bounds to the calculations for positional changes based on theinertia data. For example, the changes in magnetic field strength mayhave a certain minimum lower bound limit assuming travel directlyperpendicular to the magnetic field lines where they are mostcompressed. Given a certain change in magnetic field strength, it couldbe determined that the magnetometer had at least travelled a certainEuclidean distance in space (or changed a certain angular orientation).These changes in inclination and field strength may be utilized todetermine distance changes within certain margins of error that may besupplemented or fine-tuned according to accelerometer or gyrometer data.This data could be combined with orientation and inertial data todetermine a more precise distance travelled and relative directionaltravel of the oral hygiene device 1.

Brushing Time and Position—Visual Pattern Recognition

An optical sensor 9 and pattern 120 recognition system may be utilized,in some embodiments, to track the movement of the oral hygiene device 1.For instance, as described above, the oral hygiene device 1 may containpatterns 120 on the handle 40 or head 42 that can be recognized by avisual tracking system. The visual tracking system may be able torecognize the orientation, distance and position of a pattern 120 thatis on the oral hygiene device 1. For example, in some embodiments, thepattern 120 may be both on the head 42 and a separate pattern 120 may beon the handle 40 to allow the system to determine the position of boththe handle 40 and the head 42 of the oral hygiene device 1 to assist indetermining the orientation and movement. Visual pattern recognitionshave been utilized to detect the movement and orientation of markers,such as for example the system described in “Towards Positioning throughVisual Markers,” by Bernhard L. Ecklbauer, the disclosure of which isincorporated by reference in its entirety. For instance, imageprocessing software may identify the boundaries of the oral hygienedevice by standard boundary image identifiers, identify objects theappropriate dimensions and size for a toothbrush, and then attempt todetermine an orientation of the oral hygiene device (or determine anorientation of the pattern 120 and the oral hygiene devicerespectively).

In some embodiments, the visual pattern recognition system may also beutilized to determine the position and orientation of the human face.This will be useful to compare the relative position and orientation ofthe face and/or mouth and the oral hygiene device 1 in order todetermine the position of the oral hygiene device with respect to themouth. In some embodiments, the system may first acquire an image withthe oral hygiene device 1 at the same distance from an optical sensor 9as a human face. Additionally, an initial calibration using theorientation and size of the face may be utilized to calibrate thedistance from the camera 9 to the oral hygiene device 1, so size andmovement can be estimated. In some embodiments, since the size of theoral hygiene device 1 is known, the relative size of the face using theoral hygiene device 1 could be determined, to calibrate or estimate thejaw shape and size and modify the algorithms utilized to determine whichzone is being brushed.

For instance, the system could determine whether the oral hygiene device1 is in or near the mouth if the system can be utilized to determine theposition and orientation of the face. This will help the system validateand confirm when usage starts and stops, as the oral hygiene device 1would only be near the mouth of the user is actually using it. Rarelywould a user hold the brush near the mouth when they are not actuallybrushing their teeth. Rather, a user generally immediately washes itoff/puts down the oral hygiene device after they finish brushing.

Accordingly, the data detected by the camera can be evaluated todetermine the position, movement and orientation of an oral hygienedevice 1. For example, an oral hygiene device 1 may have a code “AB”with a circle and line as illustrated in FIG. 6 either on the back ofthe head 42, or on the handle 40, or both. In some embodiments, thepattern 120 could be any other pattern 120 that has a distinctorientation as described herein. The camera 9 may detect the pattern andthe data may be retrieved for processing and analysis. For example, thevisual data detected by the camera may be output and a processor mayevaluate the data to determine the orientation and size of the pattern120 and the relative sizes of certain parts of the pattern 120.

For instance, if the pattern is AB and it is oriented as pictured on theback of the head 42 of the oral hygiene device 1 and the data analysisdetermines that the AB is oriented upright, then it is likely that theuser is not brushing their teeth. This is because this will mean theoral hygiene device 1 itself is upright (as pictured in FIG. 6 ) whichwould be a highly unlikely position for a user to hold an oral hygienedevice 1 to brush their teeth. Rather, the longitudinal axis of an oralhygiene device 1 will generally be held in a horizontal plane duringtooth brushing. However, if the camera 9 detects data that indicates theAB on the head of the oral hygiene device 1 is sideways, it likely meanstwo things: (1) the oral hygiene device 1 is positioned that the back ofthe head 42 of the oral hygiene device 1 is facing the user because theAB is actually visible to the camera 9 and (2) the oral hygiene device 1is horizontal because the AB is oriented on its side. This, forinstance, would provide a high likelihood that the oral hygiene device 1was brushing the front incisors. This is because the camera 9 isgenerally positioned so that the user is facing the camera 9 as theybrush their teeth. Accordingly, if the AB is visible to the camera 9 andlying on its side, then the oral hygiene device 1 must be oriented withthe bristles facing the mouth (to brush the front surfaces) and brushingthe front teeth (rather than the sides) because the AB is still visibleto the front facing camera 9.

In many oral hygiene device 1 positions, however, the head 42 of theoral hygiene device 1 and any associated pattern 120 may not be visibleto the camera 9. For instance, when brushing the molars, tongue, top orbottom surfaces of the teeth, a pattern 120 on the back of the head 42would not be visible. Accordingly, in some embodiments, an additionalpattern(s) 120 could be included on an attachment 130 on the end of theoral hygiene device 1. This would allow the camera 9 to detect theorientation of the pattern 120 on the attachment 130 and/or handle 40while the oral hygiene device 1 is in inserted inside the user's mouth.For instance, in some embodiments, the attachment 130 may be a sphericalshaped bulb on the bottom of the handle 40 of the oral hygiene device 1,and would include several different patterns in different positionsaround the sphere.

This would allow the camera to detect the position and orientation ofseveral different patterns. Upon visually detecting a pattern, thesystem could determine (1) which pattern 120 is being detected, (2) thesize of the pattern 120, (3) the orientation of the pattern 120, (4) therelative size of different components of the pattern 120, (5) othercharacteristics of the pattern 120. This information could be downloadedand compared to existing data regarding the types and orientation ofpatterns 120 that are placed on the attachment 130 (and/or handle 40 orhead 42 of the oral hygiene device 1). For example, the system mayinclude a database that stores each of the different patterns 120 anddifferent size and orientation information with respect to the pattern120 in relation to the oral hygiene device 1 and distance from thecamera 9. For example, each pattern 120 may include information about ifit is found in an upright orientation, then the oral hygiene device ison its side, pointed in the mouth, etc. In other embodiments, the sizeof the pattern 120 detected may be equated to the distance of the oralhygiene device 1 from the camera 9, assuming a set distance. In someembodiments, calibration information may be further added to enhance thesystems analysis of brushing data for a particular individual and/or aparticular bathroom. Therefore, each pattern 120 may have a pixel numberassociated with each component (e.g., height/width of certain features)that may be associated with a certain distance from the camera 9. Thismay be an approximate or average, or use statistics to find thedistance/orientation of the highest probability.

Then, once this pattern information is determined, it could be comparedto calibration information or other data representing the patterns 120and respective orientations that are included on the attachment 130 forexample. If for example, an AB pattern 120 is included on the attachment130 as illustrated in FIG. 6 , with an upright orientation and facingforward in the same direction as the bristles, the if this pattern isdetected upright, it is unlikely the user is brushing their teeth.However, again, if this pattern 120 is detected with a sidewaysorientation, it is likely the user is brushing their front molars. Inanother example, if another pattern, BC is placed on the bottom of thetoothbrush, when the system detects that pattern it is likely that theoral hygiene device is being used to clean the molars. Accordingly, theorientation of the BC could then be used to determine whether it islikely the user is cleaning the tops, bottoms, or sides of the molars.In some embodiments, this could be combined with accelerometer andgyrometer data to determine which side of the mouth is being brushed.

In still another embodiment, the system may be able to determine theposition of the oral hygiene device (without a pattern) with respect tothe face, including the vector direction of its longitudinal axis.Accordingly, the system could then determine whether the oral hygienedevice is likely on the left or right side of the mouth. This could bedone using the visual recognition system as disclosed herein. In otherembodiments, the system could determine the position and orientation ofthe mouth and oral hygiene device with respect to each other. Forinstance, a mouth recognition algorithm could be used to approximate theposition of each of the corners or sides of the mouth. Additionally,there could be visual markers on the neck of the oral hygiene device 1to help the system determine which side of the mouth is being brushedwhen the user has the oral hygiene device inside the mouth.

No Pattern on Toothbrush

As illustrate in FIG. 7 and in some embodiments, the system may onlydetect the outline of the oral hygiene device 1 handle to determine anorientation of the longitudinal axis with respect to gravity withwithout utilizing a pattern 120, or markings. In other embodiments, thesystem may have additional image processing software to identifyspecific features of the oral hygiene device including the head(bristles, etc.) by utilizing classification algorithms to classifycertain portions of the image. In some embodiments, the image processingsoftware identifies the longitudinal axis, the head, the base andpotentially the bristles. From this information, the orientation andposition of the oral hygiene device may be determined. In someembodiments, the user and the mouth of the user may be identifiedutilizing similar boundary detection/shape classification software.Accordingly, the relative position of the oral hygiene device withrespect to the mouth may be determined, as explained further herein.

The inventors may utilize this system so that a customized oral hygienedevice is not necessary, and only a chip would need to be providedinside the oral hygiene device that includes motion sensors 11.Accordingly, the device could be manufactured utilizing an existing oralhygiene device designs and only implementing a chip or controller 13system that includes a motion sensor 11, antenna 5 or other wirelessconnectivity, and a memory. Then the device could send all of the datato a connected smartphone or other connected device 30 (e.g. smartmirror, etc.)

Combining Visual and Motion Data for Position Determination

The visual data may also be used to determine acceleration and thevector or direction of motion of the oral hygiene device 1 as it isbrushing the teeth. The visual data may capture the different positionsand time stamp the data to determine the amount of time recorded totravel for a first point to a second point and therefore the resultantvelocity (direction and speed) and acceleration. As described herein thepositional data may be utilized to calculate brushing positions. Thiscombined with the orientation data will be able to provide enoughpositional and orientation data to determine the position in a user'smouth and the section of the mouth that is being brushed as describedherein.

For instance, in some embodiments, the visual information from patternrecognitions or brush outlines may be utilized to recalibrate or theposition output from the accelerometer/magnetometer/gyrometer positiondetermination. In these embodiments, the visual data may be utilized torecalibrate the drift experienced by the position sensors to bring themback to a reference coordinate system. In other embodiments, the visualdata may be utilized to recalibrate the entire statistical model used todetermine position.

In some embodiments, position may be independently determined frommotion/acceleration/position sensors, and independently determined fromvisual recognition data. These two determinations may then be utilizedto determine a final position with respect to the mouth. In otherembodiments, just the visual recognition data alone may be utilized tocalculate position with respect to the mouth.

For instance, a first example would a setup with no visual pattern 120added to the oral hygiene device 1. Using data output from the motionsensor(s) 11 only, it may be challenging to distinguish, in somesituations, which side of the mouth the oral hygiene device 1 is on whenthe oral hygiene device is oriented in the same direction but could bebrushing different sections of teeth. For instance, its orientationabout an axis is identical or near identical, but its position on oneside of the face, is on the left or the right side of the mouth.

For example, an ambiguous determination could be: is the user brushingthe outer face of the teeth on the left side of the mouth, or is theuser brushing the inner face of the teeth on the right side of themouth? Even with no additional visual pattern 120, with visualinformation of the oral hygiene device and the user's face alone thesystem can make a distinction between left and right. So, thecombination of algorithms in this case is beneficial in that inertialdata is known to be insufficient to establish a relative spatialposition on a coordinate system surrounding the head. Thus, the visualinformation can be utilized to make a secondary determination of whichside of the head the brush is on, and therefore, the visual informationand the orientation information (about an axis or point or with respectto gravity) can be combined to determine which side of the head thebrush 1 is on and whether (for example the inner or outer face of theteeth are being brushed).

Another example may include an embodiment with a pattern 120 attached tothe oral hygiene device 1. For instance, a pattern 120 could be added tothe bottom of the oral hygiene device 1 and to the back of the brush 1head. With a pattern 120 however, there may be positions of the oralhygiene device 1 in which visual information alone would not be enoughto determine the location and orientation of the oral hygiene device 1with enough accuracy. For example, when the visual tags are partiallyhidden, or too small in the image resolution to be accurately detectedin size and orientation.

For example, if a user is brushing the molars, and the only visiblepattern 120 is on the oral hygiene device 1 bottom, and the resolutionsis low (e.g., because the user is a bit too far from the camera device,or because the user is using a camera that has insufficient resolution)then the system may not be able to determine with enough accuracy theangle with which the brush 1 is on the teeth. Therefore, the systemcould not determine whether the user is brushing the outside of themolars of the upper jaw or the outside of the molars on the lower jaw.

In this case, the angle difference (e.g. the angle of the longitudinalaxis of the oral hygiene device with respect to gravity because you holdthe oral hygiene device at a slightly different angle when brushing thetop or bottom molars) may be quite subtle between those two brushingpositions. However, the motion data (especially acceleration andvertical orientation) may be able to disambiguate the two potentialpositions narrowed down by the visual data and hence provide thenecessary information to if the oral hygiene device 1 is brushing theupper or lower jaw. Accordingly, the visual data alone can besupplemented with the motion data to determine the position or sectionof teeth that are brushed.

Brushing Time and Position—Analysis of Motion Data

In order to utilize the data from the sensors to estimate the amount oftime spent in each section, quadrant or other logical division of themouth, the user may need to calibrate the oral hygiene device 1. In someembodiments, calibration may take place at the factory, by the user orboth. Accordingly, calibration could be utilized to calibrate aspecifically manufactured unit, and to adapt to a particular user'sgeography, anatomy (i.e. height) and/or bathroom environment, which forexample, may include unique magnetic interferences unique heights andorientations (tabletops may not be level, etc.). For example, in someembodiments, the user may apply a colored gel to their teeth. The usercan then turn the oral hygiene device 1 in calibration mode once theoral hygiene device 1 is in position in by the teeth and brush away allof the colored gel, while the motion sensors 11, optical sensor(s) 9 andother sensors are recording data.

At this point, the user may switch off calibration to stop theacquisition of calibration data. In other embodiments, once the userturns on the calibration function, the oral hygiene device 1 woulddetect the points in time when the user has started and stoppedbrushing, including any pauses in the brushing. This calibrationprocedure provides reference positional, orientation, and motioncalibration data that can then be compared to subsequent brushes usingstatistical analysis 78, or other analysis methods to determine whethersubsequent brushes have brushed all regions of the mouth for sufficienttime.

Once the device is calibrated, or using reference data from previousbrushing or and other model users brushing, brushing data from eachusage may be compared to reference data to evaluate its quality 78 anddetermine the position of the oral hygiene device 1 in the newlyacquired data. For instance, the calibration data may be utilized toprepare a statistical model of the shape and dimensions of a specificuser's mouth and be used to create certain statistical boundaries fordetermining when the oral hygiene device 1 is brushing any certainsection of the user's mouth. Then, once a user initiates brushing asecond time, the newly recorded data could then be overlaid orcorrelated with the calibration brushing data associated with that userprofile 60, to determine whether and where there were any deficienciesin the brushing. For instance, the system 100 may determine how muchtime is spent brushing each quadrant, how many strokes, or perform otherevaluations of the brushing. Additionally, the analysis may indicatethat a user brushed too much in certain areas or did not use the correctstroke patterns generally or in specific sections of the user's mouth.

The correlation may be performed by statistical analysis, for example, aquantitative comparison of such differences can be made simply bymeasuring the Euclidean distance in the 3 xz space. Such vectors maythen be treated by using a statistical analysis, including principalcomponent analysis (PCA), hierarchical cluster analysis (HCA),regression analysis, and linear discriminant analysis. Statisticalmethods suitable for high dimensionality data may be utilized. As anexample, HCA systematically examines the distance between the data thatrepresent each type of motion or positional data. A dendrogram can thenbe generated that shows the clustering of the data from the Euclideandistances between and among the data vectors, much like an ancestraltree.

Each statistical data cluster may be representative of sensor data foreach section, quadrant or other logical division of the user's mouth.For example, the mouth may be divided into, top and bottom, with top andbottom each having a right front and left portion, and each of theright, front and left portions, having an inside, top, and outside face.Accordingly, the mouth may have 36 different sections, 10 sections, 8sections, 12 sections, or 4 sections (quadrants), top and bottom only,outer and inner only, and other logical divisions. Accordingly, usingreference data, the controller 13, or other processors in the system 100(e.g., processors in the mobile phone or servers), may calculate theconfidence interval or other value indicating the likelihood the oralhygiene device is brushing a specific section of the user's mouth, butdetermine the likelihood the strokes correlate or relate to thecalibration data in those sections. In order to acquire a referenceposition, the user may be instructed to initiate brushing at anidentical position of the mouth. That way, the data starts at a knownposition and can use that as a reference point to relate the rest of thedata using statistical analysis. For instance, the angle, orientation,calculated distance and other features from the reference position ofthe oral hygiene device 1 detected by the motion sensor 11 and/oroptical sensor 9 may indicate the likelihood each new position iscorrelated to a certain position of the mouth based on calibration data.

In other embodiments, the control system 13 or other system processors(e.g. processors in the smartphone, servers, or other components) mayprocess the statistical data and determine that the brush is brushing acertain section of the mouth if the statistical analysis shows withgreat than 95%, 85%, 80% or other suitable certainty that the brush iswithin the section of the mouth. In some embodiments, the specificity orsensitivity of the brushing statically analysis may be modified to matcha specific user's variance.

This analysis may include an output 80 that allows the control system 13or other processors to determine which brush strokes were in eachsection of the mouth or tooth set, and therefore how much time a userspend brushing each quadrant, half or tooth, or other logical divisionof the mouth and how those times compare to recommended regimes. Inother embodiments, the time at the beginning and end of entering acertain area may be recorded, or other suitable methods to determine howmuch time is spent brushing each section of the mouth.

In some embodiments, rather than detect the position of the toothbrush,the system 100 may direct the user to brush certain sections of themouth one at a time, and only determine whether or not the user isactually brushing, and count or record brushing time only while motionindicative of brushing is detected. Then, once brushing has beenrecorded in the directed section for the prescribed time, the system 100would indicate to the user that it is time to move to the next sectionof the mouth, and elapse time when the user is actually brushing. Thisprevents the user from taking breaks and thereby brushing for less thanthe optimal amount of time in each section. Accordingly, the controlsystem 13 or other processors may analyze the data from the sensors 11(or pressure sensor 10) to record the amount of time the user wasbrushing in each section while the system 100 indicated the user shouldbe brushing in that particular section.

In other embodiments, pressure exerted on the head 42 from brushing mayalso be detected by a pressure sensor 10 incorporated into oral hygienedevice 1. For example, the pressure sensor 10 may be incorporated intothe head 42, and detect a pressure associated with usage. Additionally,the pressure may be mapped to the various positional data and thereforethe controllers 13 or other processors of the system 100 may calculatethe amount of pressure utilized for brushing each area of the mouth.

Machine Learning

In some examples, the statistical analysis utilized to implement variousfeatures disclosed in the system 100 will be a machine learning orartificial intelligent algorithm. For instance, the system 100 mayprocess the available data sources from the systems 100 sensors includedas disclosed herein, and identify a position within the mouth. In otherexamples, machine learning algorithms may be utilized to: (1) identifycavities or plaque using machine vision or other combinations of data,(2) identify position in the mouth to varying degrees of granularity,(3) identify bush stroke type using the dictionary of strokes disclosedherein, (4) identify users from motion or image data (e.g. machinevision to recognize facial features), or (5) other useful applicationsto the features described herein.

Machine learning algorithms may take a variety of forms. For instance,the system 100 may utilize more basic machine learning toolsincluding 1) decision trees (“DT”), (2) Bayesian networks (“BN”), (3)artificial neural network (“ANN”), or (4) support vector machines(“SVM”). In other examples, deep learning algorithms or other moresophisticated machine learning algorithms may be used.

DT programs are generally used because of their simplicity and ease ofunderstanding. DT are classification graphs that match input data toquestions asked at each consecutive step in a decision tree. The DTprogram moves down the “branches” of the tree based on the answers tothe questions (e.g., First branch: Is the patient male? yes or no.Branch two: Is the patient having trouble urinating? yes or no. etc.).

Bayesian networks (“BN”) are based on likelihood something is true basedon given independent variables and are modeled based on probabilisticrelationships. BN are based purely on probabilistic relationships thatdetermine the likelihood of one one variable based on another or others.For example, BN can model the relationships between symptoms anddiseases. Particularly, if a patient's symptoms or biomarkers levels areknown, a BN can be used to compute the probability that a patient has aparticular disease. Thus, using an efficient BN algorithm, an inferencecan be made based on the input data. They are commonly used by themedical domain to represent reasoning under uncertain conditions for awide range of applications, including disease diagnostics, geneticcounseling, and emergency medical decision support system (MDSS) design.

Artificial neural networks (“ANN”) are computational models inspired byan animal's central nervous system. They map inputs to outputs through anetwork of nodes. However, unlike BN, in ANN the nodes do notnecessarily represent any actual variable. Accordingly, ANN may have ahidden layer of nodes that are not represented by a known variable to anobserver.

ANNs are capable of pattern recognition and have been used for themedical and diagnostics fields. Their computing methods make it easierto understand a complex and unclear process that might go on duringdiagnosis of an illness based on input data a variety of input dataincluding symptoms. While still facing steep limitations, ANN hasdemonstrated to be suitable in CDSS design and other biomedicalapplications, such as diagnosis of myocardial infarction, MDSS forleukemia management, and cancer detection.

Support vector machines (“SVM”) came about from a framework utilizing ofmachine learning statistics and vector spaces (linear algebra conceptthat signifies the number of dimensions in linear space) equipped withsome kind of limit-related structure. In some cases, they may determinea new coordinate system that easily separates inputs into twoclassifications. For example, a SVM could identify a line that separatestwo sets of points originating from different classifications of events.

They have been applied practically and are theoretically well-founded,but can sometimes be difficult to understand. SVMs have been applied toa number of biological domains, such as MDSS for the diagnosis oftuberculosis infection, tumor classification, and biomarker discovery.

However, there is a relatively new type of machine learning algorithmthat is capable of modeling very complex relationships that have a lotof variation that are called deep neural networks. Deep neural networkshave developed recently to tackle the problems of speech recognition.

In the IT industry fields, various architectures of DNN have beenproposed to tackle the problems associated with algorithms such as ANNby many researchers during the last few decades. These types of DNN areCNN (Convolutional Neural Network), RBM (Restricted Boltzmann Machine),LSTM (Long Short Term Memory) etc. They are all based on the theory ofANN. They demonstrate a better performance by overcoming theback-propagation error diminishing problem associated with ANN.

Machine Learning—Training Data

Machine learning algorithms require training data to identify thefeatures of interest that they are designed to detect. For instance,various methods may be utilized to form the machine learning modelsincluding applying randomly assigned initial weights for the network andapplying gradient descent using back propagation for deep learningalgorithms. In other examples, a neural network with one or two hiddenlayers can be used without training using this technique.

In some examples, the machine learning algorithms will be trained usinglabeled data, or data that represents certain features, specificactions, or characteristics, including a particular position in themouth, a particular brush stroke, a particular user and others. In someexamples, the training data will be pre-filtered or analyzed todetermine certain features, including various high level filters orstarting points that include motion sensing data (brush picked up, brushfacing certain direction with respect to gravity). In other examples,the data will only be labeled with the outcome and the various relevantdata may be input to train the machine learning algorithm.

For instance, to identify the position in the mouth, various machinelearning algorithms may be utilized that input various data disclosedherein, including motion (accelerometer output, gyroscope output,magnetometer output), visual data from the camera on the base 2, orother data that is relevant to the position of the oral hygiene device 1during brushing. For instance, in some embodiments, the input data fordetermining toothbrush position may only include data output from thesensors on the oral hygiene device 1. For instance, in some examples,the input data will be labeled data from the acceleration sensor alone.

In other examples, to identify position, the input data may be variouscombinations of labeled data output from the accelerometer,magnetometer, and gyroscope. In some examples, the starting and endingpoints of the input data will be quite important. In some examples, theinput data may be a combination of data output from the sensors on theoral hygiene device, and sensors on the base 2, for example, the visualrecognition or depth perception systems.

Similarly, for systems 100 that identify types of brush strokes (e.g.specific actions), the systems may utilize machine learning fromcombinations of data output from various sensors. In some examples, dataoutput from the accelerometer alone may be sufficient to identify brushstrokes. This is particularly the case because most brush strokes have aparticular acceleration pattern that is repeated, and may includeangular information with respect to gravity as disclosed herein.

In systems 100 that identify users, machine learning algorithms may beutilized to detect the face of the user, and identify the user with aunique profile. In these examples, the user may first use the systemthat includes a base 2 with a camera 56 that may take an initial pictureof the user. The user can then indicate their profile selection, and thesystem can use that picture to identify that user in the future. Overtime, the system may acquire additional photos or video of the user tomake the identification of the user more robust (more labeled trainingsets, for example).

Machine Learning—Acquisition of Training Data

In some examples, the labeling of training data may include operatorsthat review data that is recorded while a video camera captures imagesof a user. In other examples, a user prompt will tell the user (1) whereto brush, and (2) potentially the type of brush stroke (e.g. otherspecific actions) to use, to train the system. For instance, in someexamples, the system 100 may utilize a coach or other instructor with aschematic of the mouth that may highlight, light up, or otherwiseinstruct the user where to brush. These instructions may be displayed ona display of a mobile device, for instance.

Accordingly, an automatic instruction system may be utilized to acquiretraining data for a particular user (to personalize the algorithm) orfor many users. For instance, in the case where the system 100implements a mobile device 30, the system may display a schematic of theuser's dentition on the mobile device 30 screen, and indicate whichregions the user should brush in sequence, real time. Then, the systemcan record the data as labeled training data, with each set of dataindicating a certain position in the mouth.

Additionally, the system 100 may indicate the type of brush stroke touse, including circular, back and forth, or more complex strokes asdisclosed herein. Accordingly, these will all provide input trainingdata to the system 100 to learn how to identify positions in the mouthand certain brush strokes.

In some examples, however, the input data from the coach styleacquisition may be required to be filtered, have the dimensionalityreduced before feeding the labeled data into the algorithm fordeveloping a model. For instance, a principal component analysis orother dimensionality reduction method may be implemented before thelabeled data is fed into the algorithm(s).

In other examples, various filters may be utilized to filter the databefore labeling and entering the data as training data. For instance, ifa machine learning model for identifying position is being developed,the system may filter out various accelerations that include fast timechanges, or periodic accelerations that are related to brush strokesrather than position. Similarly, for brush strokes, accelerations thatare steady (e.g. gravitational) may be filtered out and only periodicaccelerations will be analyzed or fed into the system. In other examplesthe data may not be pre-filtered and the sensitive deep learningalgorithms may reliably determine position with additional data.

In other examples, the data may be pre-filtered or post-filtered (laterrejected after analysis) in case the user is performing someone actionwrong or not in accordance with the instructions displayed on the mobile30 or other device display. Additionally, various filters or techniquesmay be utilized to determine when the user is in position and complyingwith the instructions, for instance once a periodic brushing is detected(with a periodic change in acceleration for example).

Presentation of Feedback

Once the system's 100 controller(s) 13, or other processors contained inthe smartphone, servers, or other components of the system 100 haveanalyzed the usage data 78, feedback may be presented to the user 80through the speaker 50, the visual indicators 52 on the oral hygienedevice 1, or through an associated mobile device 30 or other computingdevice in data communication with the system 100. This feedback may bepresented instantaneously or available for access to check progress.

For instance, instantaneous feedback may be provided to the user 80during brushing, that includes indications by audio or visual means thatindicate how much time is left, whether to brush harder or softer,whether certain quadrants have been sufficiently brushed, and whenbrushing is completed. For example, a red light or stop sound may beproduced through the speaker 50 to indicate the brushing is completed.

Additionally, historic and average brushing times and positional datamay be presented to the user 80 on the mobile device or other computingdevice using graphs, charts, percentages and other metrics. Forinstance, the user could be presented with the average time spent perday, per quadrant, and the average days the user brushes once, twice orother times. Additionally, the average time spent brushing per tooth maybe calculated and presented to the user on the mobile device 30 or othercomputing device. A program 15 running on the mobile device may controlthe presentation of the data, using the mobile device 30 controller 13.

In some examples, the system may combine the filtered and processedposition and brush stroke identifications, and potentially tarteridentification to give holistic feedback. For instance, in someexamples, the feedback will not only be related to the time in eachposition of the mouth, but will also include the strokes used in eachposition of the mouth. In some examples, certain stroke techniques willpresent higher feedback scores to the user, or certain strokes that areused in certain positions in the mouth.

Accordingly, as show in FIGS. 9A and 9B, an application according to thepresent invention may include a mobile device 30 display 910 that mayinclude a heatmap 920 or other visual representation of the positionsthe user has brushed, the accuracy of the brushing in each position, andthe amount of time, and the type of stroke used. This information canprovide a user with multiple goals for improving brushing.

API for Dentist Integration

An API may also be developed for transferring the brushing and usagedata to a dentist for professional evaluation and feedback on brushinghabits and techniques. Furthermore, this data may be evaluated todetermine brushing techniques that result in more dental issues,including cavities and gum disease. For instance, the width or height ofthe strokes may be correlated to how high along the gum-line the brushreaches, indicating whether the gums are properly brushed. This could beanalyzed to determine whether short strokes in height lead to morefrequent instances of gum disease.

Gamification

The usage data may also be used to create games for children or familymembers to play alone or against each other, to hit certain thresholdsof usage. This can provide a unique motivational tool for users to brushtheir teeth on a regular basis. Accordingly, as delayed feedback, forexample, in the case of gum disease or cavities generally is discountedpsychologically, the more immediate and continual concrete, andquantitative feedback of the present disclosure will provide a muchbetter incentive to conform to recommend brushing regimens.

In some examples, the games will show what is inside the mouth, how tobrush well and how to train & evaluate the quality of kid's brushing. Insome examples, the user's brush to earn points based on time, quality,and quantity of brushing. For instance, the system may

Tele-Dentistry Platform

In some examples, the systems and methods disclosed herein may be incommunication with dentist professionals, to provide diagnosis on dentalconditions and feedback on brushing. This may include the cavity,plaque, and brushing position time and technique information. Forinstance, in some embodiments, a user profile may be integrated with theservers of a dentist office, or separately contain patient data added bythe dentist that includes any specific maladies or other issues apatient may have.

For instance, the dentist may add specific areas or teeth that havecavities, periodontal diseases, gingivitis, tooth decay, recedinggumlines, and other issues. Accordingly, those issues may be mapped toan individual user's dentition and change the recommended brushingstrategy presented to the user during brushing and evaluated against forproviding feedback.

In some examples, a dentist (using a computer or mobile device) mayenter into that the user has a receding gumline in a certain position inthe user's mouth. Accordingly, that data will be sent to the patient'smobile device 30 and saved with the patient profile. Accordingly, whenthe user brushes, the interface may highlight in red where the usershould brush more softly, and using a circular or other type of stroketo minimize gum damage. Thus, the user will then attempt to accommodatethat request, the feedback may indicate the pressure is too high forthat area (including real time) the stroke is wrong, too fast, etc., ortoo much time is spent in a particular area.

Accordingly, the tele-dentistry system may be able to provide clinicianlevel feedback in the brushing process. This will allow the brushingexperience to be personalized to each user.

Computer & Hardware Implementation of Disclosure

It should initially be understood that the disclosure herein may beimplemented with any type of hardware and/or software, and may be apre-programmed general purpose computing device. For example, the systemmay be implemented using a server, a personal computer, a portablecomputer, a thin client, or any suitable device or devices. Thedisclosure and/or components thereof may be a single device at a singlelocation, or multiple devices at a single, or multiple, locations thatare connected together using any appropriate communication protocolsover any communication medium such as electric cable, fiber optic cable,or in a wireless manner.

It should also be noted that the disclosure is illustrated and discussedherein as having a plurality of modules which perform particularfunctions. It should be understood that these modules are merelyschematically illustrated based on their function for clarity purposesonly, and do not necessary represent specific hardware or software. Inthis regard, these modules may be hardware and/or software implementedto substantially perform the particular functions discussed. Moreover,the modules may be combined together within the disclosure, or dividedinto additional modules based on the particular function desired. Thus,the disclosure should not be construed to limit the present invention,but merely be understood to illustrate one example implementationthereof.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someimplementations, a server transmits data (e.g., an HTML page) to aclient device (e.g., for purposes of displaying data to and receivinguser input from a user interacting with the client device). Datagenerated at the client device (e.g., a result of the user interaction)can be received from the client device at the server.

Implementations of the subject matter described in this specificationcan be implemented in a computing system that includes a back-endcomponent, e.g., as a data server, or that includes a middlewarecomponent, e.g., an application server, or that includes a front-endcomponent, e.g., a client computer having a graphical user interface ora Web browser through which a user can interact with an implementationof the subject matter described in this specification, or anycombination of one or more such back-end, middleware, or front-endcomponents. The components of the system can be interconnected by anyform or medium of digital data communication, e.g., a communicationnetwork. Examples of communication networks include a local area network(“LAN”) and a wide area network (“WAN”), an inter-network (e.g., theInternet), and peer-to-peer networks (e.g., ad hoc peer-to-peernetworks).

Implementations of the subject matter and the operations described inthis specification can be implemented in digital electronic circuitry,or in computer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Implementations of the subjectmatter described in this specification can be implemented as one or morecomputer programs, i.e., one or more modules of computer programinstructions, encoded on computer storage medium for execution by, or tocontrol the operation of, data processing apparatus. Alternatively, orin addition, the program instructions can be encoded on anartificially-generated propagated signal, e.g., a machine-generatedelectrical, optical, or electromagnetic signal that is generated toencode information for transmission to suitable receiver apparatus forexecution by a data processing apparatus. A computer storage medium canbe, or be included in, a computer-readable storage device, acomputer-readable storage substrate, a random or serial access memoryarray or device, or a combination of one or more of them. Moreover,while a computer storage medium is not a propagated signal, a computerstorage medium can be a source or destination of computer programinstructions encoded in an artificially-generated propagated signal. Thecomputer storage medium can also be, or be included in, one or moreseparate physical components or media (e.g., multiple CDs, disks, orother storage devices).

The operations described in this specification can be implemented asoperations performed by a “data processing apparatus” on data stored onone or more computer-readable storage devices or received from othersources.

The term “data processing apparatus” encompasses all kinds of apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, a system on a chip, or multipleones, or combinations, of the foregoing The apparatus can includespecial purpose logic circuitry, e.g., an FPGA (field programmable gatearray) or an ASIC (application-specific integrated circuit). Theapparatus can also include, in addition to hardware, code that createsan execution environment for the computer program in question, e.g.,code that constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, a cross-platform runtimeenvironment, a virtual machine, or a combination of one or more of them.The apparatus and execution environment can realize various differentcomputing model infrastructures, such as web services, distributedcomputing and grid computing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub-programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data, e.g., magnetic, magneto-optical disks, or optical disks.However, a computer need not have such devices. Moreover, a computer canbe embedded in another device, e.g., a mobile telephone, a personaldigital assistant (PDA), a mobile audio or video player, a game console,a Global Positioning System (GPS) receiver, or a portable storage device(e.g., a universal serial bus (USB) flash drive), to name just a few.Devices suitable for storing computer program instructions and datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

Those skilled in the art will recognize that many changes may be made tothe described embodiments without departing from the spirit and scope ofthe invention. Furthermore, those skilled in the art will also recognizethat certain embodiments described for one device or system can bereadily, or with slight modification, be included in the embodimentsdescribed for another device or system, without departing from thespirit and scope of the invention.

By way of example, the following selected embodiments are illustrativeexamples of the present disclosure.

SELECTED EMBODIMENTS

Embodiment 1. An oral hygiene system for monitoring compliance with anoral hygiene regimen comprising: an oral hygiene device comprising ahandle and a head; an optical sensor; a memory containing machinereadable medium comprising machine executable code having stored thereoninstructions for performing a method of recognizing specific actions ofthe oral hygiene device; a control system coupled to the memorycomprising one or more processors coupled to the memory, the controlsystem configured to execute the machine executable code to cause theone or more processors to: receive data output by the optical sensor;process the data using a machine learning model to determine a specificaction of the oral hygiene device represented by the data, wherein thespecific action includes an orientation and motion of the oral hygienedevice; and outputting data representing the specific action.

Embodiment 2. The oral hygiene system according to embodiment 1, whereinthe oral hygiene device comprises a pattern, and the control system isfurther configured to execute the machine executable code to cause theone or more processors to determine the specific action of the oralhygiene device based at least on an analysis of the pattern in the dataoutput by the optical sensor.

Embodiment 3. The oral hygiene system according to embodiment 1 or 2,wherein the specific action of the oral hygiene device is determinedwith respect to the user's mouth.

Embodiment 4. The oral hygiene system according to any of the precedingembodiments, wherein the machine learning model is a deep learningmodel.

Embodiment 5. The oral hygiene system according to any of the precedingembodiments, wherein the machine learning model incorporates labeledtraining data that indicates a position of the mouth.

Embodiment 6. The oral hygiene system according to any of the precedingembodiments, wherein the control system is further configured to executethe machine executable code to cause the one or more processors to inputdata output from the optical sensor and a motion sensor attached to theoral hygiene device to determine the specific action.

Embodiment 7. The oral hygiene system according to embodiment 0, whereinthe labeled training data comprises data that was automatically labeledby a user brushing their teeth according to instructions on a display ofa mobile device.

Embodiment 8. The oral hygiene system according to any of the precedingembodiments, wherein the specific action is one of: a brush stroke type,a brushing position, applying toothpaste, or brushing.

Embodiment 9. An electronic oral hygiene system for monitoring brushingand compliance with a tooth brush regime comprising: an oral hygienedevice including a handle and a head; a motion sensor configured tooutput motion data related to motion of the oral hygiene device; amemory for storing the data output by the motion sensor and in datacommunication with the motion sensor; and a control system whichdetermines the position of a head of the oral hygiene device in a user'smouth by comparing the motion data to previously recorded calibrationdata using a machine learning model.

Embodiment 10. The electronic oral hygiene system according toembodiment 9, wherein the control system utilizes a machine learningmodel to determine orientation.

Embodiment 11. The electronic oral hygiene system according toembodiment 10, wherein the machine learning model is a deep learningmodel.

Embodiment 12. The electronic oral hygiene system according toembodiment 10 or 11, wherein the machine learning model incorporateslabeled training data that indicates a position of the mouth.

Embodiment 13. The electronic oral hygiene system according toembodiment 12, wherein the labeled training data comprises data that wasautomatically labeled by a user brushing their teeth according toinstructions on a display of a mobile device.

Embodiment 14. The electronic oral hygiene system according to any ofembodiments 9 to 13, wherein the control system utilizes a machinelearning model to further determine brush stroke type.

Embodiment 15. The electronic oral hygiene system according toembodiment 14, wherein the brush stroke type comprises a circularstroke, back-and-forth stroke, or an angled stroke.

Embodiment 16. An oral hygiene system for monitoring brushing andcompliance with a tooth brush regime comprising: an oral hygiene deviceincluding a handle and a head; a sensor; a memory containing machinereadable medium comprising machine executable code having stored thereoninstructions for performing a method of determining a position andorientation of the toothbrush; and a control system coupled to thememory comprising one or more processors, the control system configuredto execute the machine executable code to cause the control system todetermine a specific action performed by the oral hygiene device basedat least on data output by the sensor comprising motion datarepresenting the motion of the oral hygiene device, wherein the specificaction is determined by processing the motion data with a machinelearning model.

Embodiment 17. The oral hygiene system according to embodiment 16,wherein the sensor is one of a: pyrometer, magnetometer, oraccelerometer.

Embodiment 18. The oral hygiene system according to embodiment 16 or 17,wherein the specific action performed by the oral hygiene device is oneof: a brush stroke type, a position in the mouth, a brush stroke type ata position in the mouth, or applying toothpaste to the oral hygienedevice.

Embodiment 19. The oral hygiene system according to any of embodiments16 to 18, wherein a first mobile device displays a schematic of a user'sdentition based on the determined specific action, and illustratesinformation relating to an amount of time spent brushing each section ofthe user's mouth and the brush stroke type used in each section of theuser's mouth.

Embodiment 20. The oral hygiene system according to embodiment 19,wherein a second mobile device in communication with the first mobiledevice receives input from a user regarding a goal for a time, quantity,or accuracy of specific actions.

Embodiment 21. The oral hygiene system according to embodiment 20,wherein the control system is configured to send a notification to thesecond mobile device if the goal is reached, and send a reward to thefirst mobile device.

Embodiment 22. A non-transitory, computer-readable storage medium havingstored thereon instructions for performing a method comprising machineexecutable code which when executed by at least one machine, causes themachine to: determine a spatial position and orientation of an oralhygiene device based at least on data output by an accelerometercomprising motion data representing the oral hygiene device using amachine learning algorithm; determine a section of a user's teeth beingbrushed by the oral hygiene device based on the motion data; and outputan indication of the user's brushing on a display based on thedetermined spatial position, orientation, and section of the user'steeth.

Embodiment 23. The non-transitory, computer-readable storage medium ofembodiment 22, wherein the section of the user's teeth comprises theoutside face of upper molars or the inside face of lower molars.

Embodiment 24. The non-transitory, computer-readable storage medium ofembodiment 22, wherein the section of the user's teeth comprises theoutside face of upper incisors or the outside face of lower incisors.

CONCLUSION

The various methods and techniques described above provide a number ofways to carry out the invention. Of course, it is to be understood thatnot necessarily all objectives or advantages described can be achievedin accordance with any particular embodiment described herein. Thus, forexample, those skilled in the art will recognize that the methods can beperformed in a manner that achieves or optimizes one advantage or groupof advantages as taught herein without necessarily achieving otherobjectives or advantages as taught or suggested herein. A variety ofalternatives are mentioned herein. It is to be understood that someembodiments specifically include one, another, or several features,while others specifically exclude one, another, or several features,while still others mitigate a particular feature by inclusion of one,another, or several advantageous features.

Furthermore, the skilled artisan will recognize the applicability ofvarious features from different embodiments. Similarly, the variouselements, features and steps discussed above, as well as other knownequivalents for each such element, feature or step, can be employed invarious combinations by one of ordinary skill in this art to performmethods in accordance with the principles described herein. Among thevarious elements, features, and steps some will be specifically includedand others specifically excluded in diverse embodiments.

Although the application has been disclosed in the context of certainembodiments and examples, it will be understood by those skilled in theart that the embodiments of the application extend beyond thespecifically disclosed embodiments to other alternative embodimentsand/or uses and modifications and equivalents thereof.

In some embodiments, the terms “a” and “an” and “the” and similarreferences used in the context of describing a particular embodiment ofthe application (especially in the context of certain of the followingclaims) can be construed to cover both the singular and the plural. Therecitation of ranges of values herein is merely intended to serve as ashorthand method of referring individually to each separate valuefalling within the range. Unless otherwise indicated herein, eachindividual value is incorporated into the specification as if it wereindividually recited herein. All methods described herein can beperformed in any suitable order unless otherwise indicated herein orotherwise clearly contradicted by context. The use of any and allexamples, or exemplary language (for example, “such as”) provided withrespect to certain embodiments herein is intended merely to betterilluminate the application and does not pose a limitation on the scopeof the application otherwise claimed. No language in the specificationshould be construed as indicating any non-claimed element essential tothe practice of the application.

Certain embodiments of this application are described herein. Variationson those embodiments will become apparent to those of ordinary skill inthe art upon reading the foregoing description. It is contemplated thatskilled artisans can employ such variations as appropriate, and theapplication can be practiced otherwise than specifically describedherein. Accordingly, many embodiments of this application include allmodifications and equivalents of the subject matter recited in theclaims appended hereto as permitted by applicable law. Moreover, anycombination of the above-described elements in all possible variationsthereof is encompassed by the application unless otherwise indicatedherein or otherwise clearly contradicted by context.

Particular implementations of the subject matter have been described.Other implementations are within the scope of the following claims. Insome cases, the actions recited in the claims can be performed in adifferent order and still achieve desirable results. In addition, theprocesses depicted in the accompanying figures do not necessarilyrequire the particular order shown, or sequential order, to achievedesirable results.

All patents, patent applications, publications of patent applications,and other material, such as articles, books, specifications,publications, documents, things, and/or the like, referenced herein arehereby incorporated herein by this reference in their entirety for allpurposes, excepting any prosecution file history associated with same,any of same that is inconsistent with or in conflict with the presentdocument, or any of same that may have a limiting affect as to thebroadest scope of the claims now or later associated with the presentdocument. By way of example, should there be any inconsistency orconflict between the description, definition, and/or the use of a termassociated with any of the incorporated material and that associatedwith the present document, the description, definition, and/or the useof the term in the present document shall prevail.

In closing, it is to be understood that the embodiments of theapplication disclosed herein are illustrative of the principles of theembodiments of the application. Other modifications that can be employedcan be within the scope of the application. Thus, by way of example, butnot of limitation, alternative configurations of the embodiments of theapplication can be utilized in accordance with the teachings herein.Accordingly, embodiments of the present application are not limited tothat precisely as shown and described.

1.-24. (canceled)
 25. A method for detecting dental caries associatedwith teeth in a mouth of a person, the method comprising: collecting,via an imaging device, image data associated with the mouth of theperson, the imaging device being located on a head of an oral hygienedevice; collecting, via a depth perception device, depth data associatedwith the mouth of the person; receiving, by a control system from theimaging device, the image data; analyzing, by the control system, theimage data; and detecting, based on the analyzing the image data and thedepth data, a presence of one or more dental caries associated with theteeth in the mouth of the person.
 26. The method of claim 25, furthercomprising: generating, by the control system based on the analyzing theimage data, a brushing quality metric associated with the mouth of theperson.
 27. The method of claim 25, further comprising: generating, bythe control system based on the analyzing of the image data, a toothquality metric associated with the mouth of the person, the toothquality metric including demineralization, dental decay, dentalabnormalities, tartar levels, plaque levels, or any combination thereof.28. The method of claim 27, further comprising: generating, based on thetooth quality metric associated with the mouth of the person, a toothquality metric map of the mouth of the person.
 29. The method of claim28, wherein the tooth quality metric is a tartar level, and wherein thetooth quality metric map of the mouth of the person is a tartar map ofthe mouth of the person.
 30. The method of claim 25, wherein theanalyzing the image data comprises analyzing wavelengths of lightreflected off surfaces in the mouth of the person.
 31. The method ofclaim 30, wherein the analyzing the wavelengths of light is based on anamount of light reflected at different wavelengths, the wavelengths ofthe light reflected, or any combination thereof.
 32. The method of claim25, wherein the control system generates the metric associated with themouth of the person based on a machine learning algorithm.
 33. Themethod of claim 25, wherein the depth perception device comprises: alight emitting device, wherein the light emitting device is configuredto project a light grid within the mouth of the person; and a detector,wherein the detector is configured to detect distortions in the lightgrid; wherein the depth data is based on the distortions in the lightgrid.
 34. The method of claim 33, wherein the depth data is used to oneor both of determine outlines of objects and generate a map of the mouthof the person.
 35. An oral hygiene system, the oral hygiene systemcomprising: an imaging device configured to collect image dataassociated with a mouth of a person; a depth perception deviceconfigured to collect depth data associated with the mouth of theperson; and a mobile device, wherein the mobile device comprises: a userinput device configured to receive user input from the person; a controlsystem configured to: receive, from the imaging device, the image data;analyze the image data; and detect, based on the analysis of the imagedata and the depth data, a presence of one or more dental cariesassociated with teeth in the mouth of the person; and a display deviceconfigured to present an indication associated with the one or moredental caries associated with the teeth in the mouth of the user. 36.The oral hygiene system of claim 35, wherein the analysis of the imagedata comprises analyzing wavelengths of light reflected off of surfacesin the mouth of the user.
 37. The oral hygiene system of claim 36,wherein the analyzing the wavelengths of light is based on an amount oflight reflected at different wavelengths, the wavelengths of the lightreflected, or any combination thereof.
 38. The oral hygiene device ofclaim 35, wherein the control system is further configured to: generate,based on the analysis of the image data and the depth data, a brushingquality metric associated with the mouth of the person.
 39. The oralhygiene device of claim 35, wherein the control system is furtherconfigured to: generate, based on the analysis of the image data and thedepth data, a tooth quality metric associated with the mouth of theuser, the tooth quality metric including demineralization, dental decay,dental abnormalities, tartar levels, plaque levels, or any combinationthereof.
 40. The oral hygiene device of claim 39, wherein the controlsystem is further configured to: generate, based on the tooth qualitymetric associated with the mouth of the user, a tooth quality metric mapof the mouth of the user.
 41. The oral hygiene system of claim 40,wherein the tooth quality metric is a tartar level, and wherein thetooth quality metric map of the mouth of the user is a tartar map of themouth of the user.
 42. The oral hygiene system of claim 35, wherein thecontrol system detects the one or more dental caries associated withteeth in the mouth of the user based on a machine learning algorithm.43. The method of claim 35, wherein the depth perception devicecomprises: a light emitting device, wherein the light emitting device isconfigured to project a light grid within the mouth of the person; and adetector, wherein the detector is configured to detect distortions inthe light grid; wherein the depth data is based on the distortions inthe light grid.
 44. The method of claim 43, wherein the depth data isused to one or both of determine outlines of objects and generate a mapof the mouth of the person.